Bug 1215008 - Cannot start docker: firewalld[661]: WARNING: COMMAND_FAILED: INVALID_IPV: 'ipv4' is not a valid backend or is unavailable
Summary: Cannot start docker: firewalld[661]: WARNING: COMMAND_FAILED: INVALID_IPV: 'i...
Status: NEW
Alias: None
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Containers (show other bugs)
Version: Current
Hardware: Other Other
: P5 - None : Normal (vote)
Target Milestone: ---
Assignee: Mohd Saquib
QA Contact: E-mail List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-09-05 14:07 UTC by Felix Niederwanger
Modified: 2023-09-07 14:01 UTC (History)
4 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments
lsmod before and after fw restart, journal (1.40 KB, application/x-xz)
2023-09-07 13:27 UTC, Martin Loviska
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Felix Niederwanger 2023-09-05 14:07:19 UTC
On a fresh Tumbleweed VM using the Minimal-VM image, docker fails to start

> # zypper in docker
> # systemctl start docker
> Job for docker.service failed because the control process exited with error code.
> See "systemctl status docker.service" and "journalctl -xeu docker.service" for details.
> # journalctl -e
> ...
> Sep 05 14:06:07 macflurry systemd[1]: Starting Docker Application Container Engine...
> Sep 05 14:06:08 macflurry dockerd[4517]: time="2023-09-05T14:06:08.012980631Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmap>
> Sep 05 14:06:08 macflurry dockerd[4517]: time="2023-09-05T14:06:08.013620069Z" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper no>
> Sep 05 14:06:08 macflurry dockerd[4517]: time="2023-09-05T14:06:08.014202595Z" level=error msg="failed to initialize a tracing processor \"otlp\"" error="no OpenTelemetry en>
> Sep 05 14:06:08 macflurry systemd[1]: var-lib-docker-overlay2-metacopy\x2dcheck404691-merged.mount: Deactivated successfully.
> Sep 05 14:06:08 macflurry firewalld[661]: WARNING: COMMAND_FAILED: INVALID_IPV: 'ipv4' is not a valid backend or is unavailable
> Sep 05 14:06:08 macflurry firewalld[661]: WARNING: COMMAND_FAILED: INVALID_IPV: 'ipv4' is not a valid backend or is unavailable
> ...
Comment 1 Felix Niederwanger 2023-09-05 14:14:06 UTC
Interestingly, after a reboot, the VM can run docker without any issues.
Comment 2 Martin Loviska 2023-09-06 13:23:13 UTC
I have reproduced the same issue, although it can be avoided if firewalld is restarted before starting docker for the first time.

Installed packages:
2023-09-06 13:12:43|install|catatonit|0.1.7-1.7|x86_64||repo-oss|
2023-09-06 13:12:46|install|e2fsprogs|1.47.0-1.3|x86_64||repo-oss|
2023-09-06 13:12:47|install|libip6tc2|1.8.9-1.4|x86_64||repo-oss|
2023-09-06 13:12:49|install|libnfnetlink0|1.0.2-1.7|x86_64||repo-oss|
2023-09-06 13:12:50|install|runc|1.1.8-1.2|x86_64||repo-oss|
2023-09-06 13:12:51|install|libnetfilter_conntrack3|1.0.9-1.7|x86_64||repo-oss|
2023-09-06 13:12:55|install|containerd|1.6.21-1.3|x86_64||repo-oss|
2023-09-06 13:12:56|install|xtables-plugins|1.8.9-1.4|x86_64||repo-oss|
2023-09-06 13:12:56|install|iptables|1.8.9-1.4|x86_64||repo-oss|

1) restart firewalld
2) start docker
Comment 3 Martin Loviska 2023-09-07 13:27:46 UTC
Created attachment 869359 [details]
lsmod before and after fw restart, journal

Uploading journal and lsmod diff before and after firewall restart.

====
localhost:~ # diff lsmod_firstboot.txt lsmod_after_restart_fw
1a2,26
> xt_conntrack           12288  1
> xt_MASQUERADE          16384  1
> nf_conntrack_netlink    61440  0
> xfrm_user              61440  1
> xfrm_algo              16384  1 xfrm_user
> xt_addrtype            12288  2
> br_netfilter           32768  0
> bridge                434176  1 br_netfilter
> stp                    12288  1 bridge
> llc                    16384  2 bridge,stp
> overlay               200704  0
> ip6table_nat           12288  0
> ip6table_mangle        12288  0
> ip6table_raw           12288  0
> ip6table_security      12288  0
> iptable_nat            12288  1
> iptable_mangle         12288  0
> iptable_raw            12288  0
> iptable_security       12288  0
> ip6table_filter        12288  0
> ip6_tables             36864  5 ip6table_filter,ip6table_raw,ip6table_nat,ip6table_mangle,ip6table_security
> iptable_filter         12288  1
> bpfilter               12288  0
> algif_hash             16384  0
> af_alg                 36864  1 algif_hash
13,14c38,39
< nf_nat                 61440  1 nft_chain_nat
< nf_conntrack          204800  2 nf_nat,nft_ct
---
> nf_nat                 61440  4 ip6table_nat,nft_chain_nat,iptable_nat,xt_MASQUERADE
> nf_conntrack          204800  5 xt_conntrack,nf_nat,nft_ct,nf_conntrack_netlink,xt_MASQUERADE
19c44
< nfnetlink              20480  2 nf_tables
---
> nfnetlink              20480  4 nf_conntrack_netlink,nf_tables
32,33c57,58
< ip_tables              36864  0
< x_tables               61440  1 ip_tables
---
> ip_tables              36864  5 iptable_filter,iptable_security,iptable_raw,iptable_nat,iptable_mangle
> x_tables               61440  15 ip6table_filter,xt_conntrack,ip6table_raw,iptable_filter,iptable_security,ip6table_nat,xt_addrtype,ip6_tables,iptable_raw,ip_tables,iptable_nat,ip6table_mangle,ip6table_security
,xt_MASQUERADE,iptable_mangle
===
Comment 4 Fabian Vogt 2023-09-07 13:53:44 UTC
(In reply to Martin Loviska from comment #2)
> I have reproduced the same issue, although it can be avoided if firewalld is
> restarted before starting docker for the first time.
> 
> Installed packages:
> 2023-09-06 13:12:56|install|xtables-plugins|1.8.9-1.4|x86_64||repo-oss|
> 2023-09-06 13:12:56|install|iptables|1.8.9-1.4|x86_64||repo-oss|

That's it.

On startup, firewalld checks whether ip{4,6}tables is available. In this case it starts before iptables is even installed, so it sets self.ip4tables_enabled = False. Installation of iptables doesn't change this value, so until a restart of firewalld, it can't be used to create iptables rules. Docker uses iptables passthrough with firewalld though.