Bug 1226444 - conntrackd "inject-upd2: Device or resource busy"
Summary: conntrackd "inject-upd2: Device or resource busy"
Status: NEW
Alias: None
Product: openSUSE Distribution
Classification: openSUSE
Component: Network (show other bugs)
Version: Leap 15.6
Hardware: x86-64 Other
: P5 - None : Normal (vote)
Target Milestone: ---
Assignee: Adam Majer
QA Contact: E-mail List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-06-17 17:33 UTC by Georg Pfuetzenreuter
Modified: 2024-07-12 15:22 UTC (History)
2 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Georg Pfuetzenreuter 2024-06-17 17:33:58 UTC
Hi,

we run a pair of openSUSE Leap machines in our openSUSE infrastructure acting as firewalls and utilize conntrackd on them.

Occasionally, messages such as the following are observed:

```
Jun 17 17:21:40 asgard2 conntrackd[5548]: [Mon Jun 17 17:21:40 2024] (pid=5548) [ERROR] inject-upd2: Device or resource busy
Jun 17 17:21:40 asgard2 conntrack-tools[5548]: inject-upd2: Device or resource busy
Jun 17 17:21:40 asgard2 conntrack-tools[5548]: tcp      6 12000 ESTABLISHED src=2a07:de40:b27e:1204::11 dst=2a07:de40:b27e:1203::b47 sport=48800 dport=80 [ASSURED]
```

When it happens when switching load from one to the other node, this is sometimes accompanied by connection hickups.

Any ideas what to do about this?
I only found one upstream thread without any responses about that error message.

We recently upgraded to 15.6 but it already happened on 15.5.

Below is some output which might be useful.

From both nodes:

```
# rpm -qa '*conn*'
libnetfilter_conntrack3-1.0.7-1.38.x86_64
conntrack-tools-1.4.5-1.46.x86_64
conntrackd-1.4.5-1.46.x86_64
```

From the currently passive node:

```
asgard2 (Firewall, Router):~ # conntrackd -i |wc -l
44248

asgard2 (Firewall, Router):~ # wc -l /proc/net/nf_conntrack
37442 /proc/net/nf_conntrack

asgard2 (Firewall, Router):~ # conntrackd -s
cache internal:
current active connections:            44190
connections created:                   99281    failed:            0
connections updated:                   45992    failed:            0
connections destroyed:                 55091    failed:            0

external inject:
connections created:                  475586    failed:            1
connections updated:                 2520789    failed:            2
connections destroyed:                 23557    failed:            0

traffic processed:
                   0 Bytes                         0 Pckts

multicast traffic (active device=os-asgard):
           152972908 Bytes sent            280619092 Bytes recv
              129771 Pckts sent              1859362 Pckts recv
                   0 Error send                    0 Error recv

message tracking:
                   0 Malformed msgs              3133984 Lost msgs
```

From the currently active node:


```
asgard1 (Firewall, Router):~ # conntrackd -i |wc -l
75578

asgard1 (Firewall, Router):~ # wc -l /proc/net/nf_conntrack
37738 /proc/net/nf_conntrack

asgard1 (Firewall, Router):~ # conntrackd -s
cache internal:
current active connections:            72576
connections created:               589946516    failed:    264642288
connections updated:              3554034767    failed:            0
connections destroyed:             589873940    failed:            0

external inject:
connections created:                94995492    failed:            2
connections updated:               138274186    failed:            1
connections destroyed:               2821975    failed:            0

traffic processed:
                   0 Bytes                         0 Pckts

multicast traffic (active device=os-asgard):
       1056753212056 Bytes sent          27692063956 Bytes recv
          3161185406 Pckts sent             45953693 Pckts recv
                   0 Error send                    0 Error recv

message tracking:
                   0 Malformed msgs           1624879339 Lost msgs
```
Comment 1 Georg Pfuetzenreuter 2024-07-12 14:51:33 UTC
Occasionally we also get "File exists" messages in between the "Device or resource busy" ones:

```
Jul 12 13:17:38 asgard1 conntrackd[5449]: [Fri Jul 12 13:17:38 2024] (pid=5449) [ERROR] inject-add2: File exists
Jul 12 13:17:38 asgard1 conntrack-tools[5449]: inject-add2: File exists
Jul 12 13:17:38 asgard1 conntrack-tools[5449]: tcp      6 60 SYN_RECV src=2a07:de40:b27e:1205::a dst=2a07:de40:b250:131:10:151:131:32 sport=47226 dport=873
```
Comment 2 Jan Engelhardt 2024-07-12 15:22:58 UTC
I do not have that much insight into conntrackd.

The "inject2-add" message was reworked in http://git.netfilter.org/conntrack-tools/commit/?id=592bb1686053cdb5cacdb1d6266d64ce976d7bf7 . The commentary posted along the commit may give some hints as to what to do next.

Perhaps you can try a newer version, e.g. 1.4.8 from security:netfilter. If that remedies the issue, we can ponder submitting it for Leap.

If all else fails, please contact upstream.