Bug 1217590

Summary: podman fails to run containers when using hybrid cgroup hierarchy
Product: [openSUSE] openSUSE Tumbleweed Reporter: Ricardo Branco <rbranco>
Component: ContainersAssignee: Dan Čermák <dcermak>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: dcermak, felix.niederwanger, pdostal, rbranco
Version: Current   
Target Milestone: ---   
Hardware: Other   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Ricardo Branco 2023-11-28 12:35:08 UTC
OS: openSUSE MicroOS 20231126

After adding "systemd.unified_cgroup_hierarchy=0" to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub and running `transactional-update grub.cfg` and rebooting I try to run a container like this:

# podman run --rm busybox ls
Error: OCI runtime error: crun: {"msg":"read from sync socket","level":"error","time":"2023-11-28T12:24:44.558897Z"}

The equivalent docker command runs without issues.

I didn't open a bug upstream because hybrid cgroup hierarchy is not supported by either podman or docker, but we plan to support it in SLES 15-SP6 when we switch to unified by default:

https://jira.suse.com/browse/PED-5849

# podman version
Client:       Podman Engine
Version:      4.7.2
API Version:  4.7.2
Go Version:   go1.21.4
Built:        Wed Nov  8 01:00:00 2023
OS/Arch:      linux/amd64

# podman info
host:
  arch: amd64
  buildahVersion: 1.32.0
  cgroupControllers:
  - cpuset
  - cpu
  - cpuacct
  - blkio
  - memory
  - devices
  - freezer
  - net_cls
  - perf_event
  - net_prio
  - hugetlb
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v1
  conmon:
    package: conmon-2.1.8-2.1.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.8, commit: unknown'
  cpuUtilization:
    idlePercent: 99.3
    systemPercent: 0.42
    userPercent: 0.28
  cpus: 2
  databaseBackend: boltdb
  distribution:
    distribution: opensuse-microos
    version: "20231126"
  eventLogger: journald
  freeLocks: 2048
  hostname: microos
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.6.2-1-default
  linkmode: dynamic
  logDriver: journald
  memFree: 3313627136
  memTotal: 4111618048
  networkBackend: cni
  networkBackendInfo:
    backend: cni
    dns: {}
    package: |-
      cni-1.1.2-3.1.x86_64
      cni-plugins-1.3.0-2.1.x86_64
    path: /usr/libexec/cni
  ociRuntime:
    name: crun
    package: crun-1.11.1-1.1.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.11.1
      commit: 1084f9527c143699b593b44c23555fb3cc4ff2f3
      rundir: /run/user/0/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: ""
    package: ""
    version: ""
  remoteSocket:
    exists: false
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /etc/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.2-1.1.x86_64
    version: |-
      slirp4netns version 1.2.2
      commit: 0ee2d87523e906518d34a6b423271e4826f71faf
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 5
      libseccomp: 2.5.4
  swapFree: 0
  swapTotal: 0
  uptime: 0h 18m 44.00s
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.opensuse.org
  - registry.suse.com
  - docker.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 21450698752
  graphRootUsed: 1308110848
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Supports shifting: "true"
    Supports volatile: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 2
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.7.2
  Built: 1699401600
  BuiltTime: Wed Nov  8 01:00:00 2023
  GitCommit: ""
  GoVersion: go1.21.4
  Os: linux
  OsArch: linux/amd64
  Version: 4.7.2
Comment 1 Dan Čermák 2023-11-28 12:52:53 UTC
The switch didn't work:

(In reply to Ricardo Branco from comment #0)
>   cgroupManager: systemd
>   cgroupVersion: v1

podman still thinks you're running with cgroups v1.

Your installation also appears to be quite old (you're still using boltdb & CNI instead of sqlite & netavark). If possible (i.e. you're fine with loosing *all* containers and settings), can you run a `podman system reset` and retry?
Comment 2 Ricardo Branco 2023-11-28 13:19:47 UTC
(In reply to Dan Čermák from comment #1)
> The switch didn't work:
> 
> (In reply to Ricardo Branco from comment #0)
> >   cgroupManager: systemd
> >   cgroupVersion: v1
> 
> podman still thinks you're running with cgroups v1.

I believe the code for both docker and podman only shows v1 and v2 when there are actually 3 modes:

- v1 (legacy)
- hybrid (v1 + v2), shown as v1 by `podman info`.
- unified (v2)

When I run the command suggested in https://unix.stackexchange.com/questions/480747/how-to-find-out-if-systemd-uses-legacy-hybrid-or-unified-mode-cgroupsv1-vs-cgr I get:

# [ $(stat -fc %T /sys/fs/cgroup/) = "cgroup2fs" ] && echo "unified" || ( [ -e /sys/fs/cgroup/unified/ ] && echo "hybrid" || echo "legacy")
hybrid

> 
> Your installation also appears to be quite old (you're still using boltdb &
> CNI instead of sqlite & netavark). If possible (i.e. you're fine with
> loosing *all* containers and settings), can you run a `podman system reset`
> and retry?


It's a fresh installation... It continues failing after a reset.

The weird thing is that other tests passed:

https://openqa.opensuse.org/tests/3762271
Comment 3 Dan Čermák 2023-12-01 14:00:40 UTC
This is a segfault in crun that has been fixed via https://github.com/containers/crun/pull/1341

I'll submit a fix later today.
Comment 4 Dan Čermák 2023-12-01 14:03:06 UTC
https://build.opensuse.org/request/show/1130251
Comment 5 Ricardo Branco 2024-03-25 19:07:12 UTC
Closing as fixed.

Tested with 20240306 running podman 4.9.3