Bug 1212483 - NetworkManager crashes frequently
Summary: NetworkManager crashes frequently
Status: NEW
Alias: None
Product: openSUSE Distribution
Classification: openSUSE
Component: Network (show other bugs)
Version: Leap 15.5
Hardware: Other Other
: P5 - None : Normal (vote)
Target Milestone: ---
Assignee: Jonathan Kang
QA Contact: E-mail List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-06-18 09:56 UTC by Andreas Schwab
Modified: 2024-03-22 02:02 UTC (History)
0 users

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---

Attachments
/var/log/NetworkManager (966.86 KB, text/x-log)
2023-07-29 07:46 UTC, Andreas Schwab 		Details
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Andreas Schwab 2023-06-18 09:56:46 UTC 
Core was generated by `/usr/sbin/NetworkManager --no-daemon'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000559e95354a35 in ipv6_dad_timeout (user_data=0x559e95a730f0)
    at ../src/core/dhcp/nm-dhcp-client.c:606
606         nm_clear_g_source_inst(&priv->v6.dad_timeout_source);
(gdb) bt
#0  0x0000559e95354a35 in ipv6_dad_timeout (
    user_data=user_data@entry=0x559e95a730f0)
    at ../src/core/dhcp/nm-dhcp-client.c:606
#1  0x00007fb27117337d in g_timeout_dispatch (source=0x559e95b028a0, 
    callback=0x559e95354a10 <ipv6_dad_timeout>, user_data=0x559e95a730f0)
    at ../glib/gmain.c:4933
#2  0x00007fb27117282b in g_main_dispatch (context=0x559e959bd8f0)
    at ../glib/gmain.c:3381
#3  g_main_context_dispatch (context=context@entry=0x559e959bd8f0)
    at ../glib/gmain.c:4099
#4  0x00007fb271172bd0 in g_main_context_iterate (context=0x559e959bd8f0, 
    block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>)
    at ../glib/gmain.c:4175
#5  0x00007fb271172ea2 in g_main_loop_run (loop=0x559e959bd9e0)
    at ../glib/gmain.c:4373
#6  0x0000559e9534a9a1 in main (argc=<optimized out>, argv=<optimized out>)
    at ../src/core/main.c:509
(gdb) p priv
$1 = (NMDhcpClientPrivate *) 0x0
Comment 1 Andreas Schwab 2023-07-02 06:47:25 UTC 
(gdb) bt
#0  0x0000563f3c8b4a35 in ipv6_dad_timeout (
    user_data=user_data@entry=0x563f3db6f240)
    at ../src/core/dhcp/nm-dhcp-client.c:606
#1  0x00007fd88889e37d in g_timeout_dispatch (source=0x563f3db414e0, 
    callback=0x563f3c8b4a10 <ipv6_dad_timeout>, user_data=0x563f3db6f240)
    at ../glib/gmain.c:4933
#2  0x00007fd88889d82b in g_main_dispatch (context=0x563f3dac98f0)
    at ../glib/gmain.c:3381
#3  g_main_context_dispatch (context=context@entry=0x563f3dac98f0)
    at ../glib/gmain.c:4099
#4  0x00007fd88889dbd0 in g_main_context_iterate (context=0x563f3dac98f0, 
    block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>)
    at ../glib/gmain.c:4175
#5  0x00007fd88889dea2 in g_main_loop_run (loop=0x563f3dac99e0)
    at ../glib/gmain.c:4373
#6  0x0000563f3c8aa9a1 in main (argc=<optimized out>, argv=<optimized out>)
    at ../src/core/main.c:509
(gdb) p/x *self
$1 = {parent = {g_type_instance = {g_class = 0xaaaaaaaaaaaaaaaa}, 
    ref_count = 0xaaaaaaaa, qdata = 0xaaaaaaaaaaaaaaaa}, 
  _priv = 0xaaaaaaaaaaaaaaaa}
Comment 2 Jonathan Kang 2023-07-04 12:09:37 UTC 
Thanks for the report, Andreas.

A detailed NetworkManager logs would be helpful. To collect this, add the following to /etc/NetworkManager/NetworkManager.conf(if exists), restart NetworkManager, reproduce this issue and attach the output of "sudo journalctl -b -u NetworkManager" here.

> [logging]
> level=trace
Comment 3 Andreas Schwab 2023-07-05 05:35:39 UTC 
WTF?

(gdb) bt
#0  l3_cfg_notify_cb (l3cfg=0x564dd7a91400, notify_data=0x7ffe61e515d0, 
    self=0x564dd7a88220) at ../src/core/dhcp/nm-dhcp-client.c:774
#1  0x00007fd87c549973 in g_closure_invoke ()
   from /usr/lib64/libgobject-2.0.so.0
#2  0x00007fd87c55da7e in ?? () from /usr/lib64/libgobject-2.0.so.0
#3  0x00007fd87c567684 in g_signal_emit_valist ()
   from /usr/lib64/libgobject-2.0.so.0
#4  0x00007fd87c56811f in g_signal_emit () from /usr/lib64/libgobject-2.0.so.0
#5  0x0000564dd6917158 in _nm_l3cfg_emit_signal_notify (
    self=self@entry=0x564dd7a91400, 
    notify_data=notify_data@entry=0x7ffe61e515d0) at ../src/core/nm-l3cfg.c:609
#6  0x0000564dd6918e00 in _nm_l3cfg_notify_platform_change_on_idle (
    obj_type_flags=2, self=0x564dd7a91400) at ../src/core/nm-l3cfg.c:1332
#7  _platform_signal_on_idle_cb (user_data=user_data@entry=0x564dd79efb00)
    at ../src/core/nm-netns.c:208
#8  0x00007fd87c4514d7 in g_idle_dispatch (source=0x564dd7b00e50, 
    callback=0x564dd6918cd0 <_platform_signal_on_idle_cb>, 
    user_data=0x564dd79efb00) at ../glib/gmain.c:5897
#9  0x00007fd87c45582b in g_main_dispatch (context=0x564dd79c6b00)
    at ../glib/gmain.c:3381
#10 g_main_context_dispatch (context=context@entry=0x564dd79c6b00)
    at ../glib/gmain.c:4099
#11 0x00007fd87c455bd0 in g_main_context_iterate (context=0x564dd79c6b00, 
    block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>)
    at ../glib/gmain.c:4175
#12 0x00007fd87c455ea2 in g_main_loop_run (loop=0x564dd79c6bf0)
    at ../glib/gmain.c:4373
#13 0x0000564dd674c9a1 in main (argc=<optimized out>, argv=<optimized out>)
    at ../src/core/main.c:509
(gdb) l
769                                      }));
770                     }
771                 } else {
772                     _LOGD("decline the lease");
773                     if (!_dhcp_client_decline(self, priv->l3cd, "DAD failed", &error))
774                         _LOGD("decline failed: %s", error->message);
775                 }
776             }
777         }
778
(gdb) p error
$3 = (GError *) 0x0
Comment 4 Jonathan Kang 2023-07-12 01:08:49 UTC 
A test package is available in this repo. Can you try NetworkManager in that repo and see whether it fixes the issue?

* https://mirrorcache-us.opensuse.org/repositories/home:/JonathanKang:/branches:/SUSE:/SLE-15-SP5:/GA/standard/
Comment 5 Andreas Schwab 2023-07-16 08:47:19 UTC 
Why did you disable debuginfo?
Comment 6 Jonathan Kang 2023-07-17 07:22:11 UTC 
(In reply to Andreas Schwab from comment #5)
> Why did you disable debuginfo?

Just enabled it. obs defaults to disabling debuginfo.
Comment 7 Andreas Schwab 2023-07-29 07:46:29 UTC 
Created attachment 868496 [details]
/var/log/NetworkManager

(gdb) bt
#0  0x0000555b2e2e0a35 in ipv6_dad_timeout (
    user_data=user_data@entry=0x555b2f23ef20)
    at ../src/core/dhcp/nm-dhcp-client.c:606
#1  0x00007f51ca1b737d in g_timeout_dispatch (source=0x555b2f37d7f0, 
    callback=0x555b2e2e0a10 <ipv6_dad_timeout>, user_data=0x555b2f23ef20)
    at ../glib/gmain.c:4933
#2  0x00007f51ca1b682b in g_main_dispatch (context=0x555b2f21fb00)
    at ../glib/gmain.c:3381
#3  g_main_context_dispatch (context=context@entry=0x555b2f21fb00)
    at ../glib/gmain.c:4099
#4  0x00007f51ca1b6bd0 in g_main_context_iterate (context=0x555b2f21fb00, 
    block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>)
    at ../glib/gmain.c:4175
#5  0x00007f51ca1b6ea2 in g_main_loop_run (loop=0x555b2f21fbf0)
    at ../glib/gmain.c:4373
#6  0x0000555b2e2d69a1 in main (argc=<optimized out>, argv=<optimized out>)
    at ../src/core/main.c:509
(gdb) l
601     ipv6_dad_timeout(gpointer user_data)
602     {
603         NMDhcpClient        *self = user_data;
604         NMDhcpClientPrivate *priv = NM_DHCP_CLIENT_GET_PRIVATE(self);
605
606         nm_clear_g_source_inst(&priv->v6.dad_timeout_source);
607
608         _emit_notify(
609             self,
610             &((NMDhcpClientNotifyData){
(gdb) p priv
$1 = (NMDhcpClientPrivate *) 0x0
Comment 8 Jonathan Kang 2023-07-31 01:48:21 UTC 
(In reply to Andreas Schwab from comment #7)
> Created attachment 868496 [details]
> /var/log/NetworkManager
> 
> (gdb) bt
> #0  0x0000555b2e2e0a35 in ipv6_dad_timeout (
>     user_data=user_data@entry=0x555b2f23ef20)
>     at ../src/core/dhcp/nm-dhcp-client.c:606
> #1  0x00007f51ca1b737d in g_timeout_dispatch (source=0x555b2f37d7f0, 
>     callback=0x555b2e2e0a10 <ipv6_dad_timeout>, user_data=0x555b2f23ef20)
>     at ../glib/gmain.c:4933
> #2  0x00007f51ca1b682b in g_main_dispatch (context=0x555b2f21fb00)
>     at ../glib/gmain.c:3381
> #3  g_main_context_dispatch (context=context@entry=0x555b2f21fb00)
>     at ../glib/gmain.c:4099
> #4  0x00007f51ca1b6bd0 in g_main_context_iterate (context=0x555b2f21fb00, 
>     block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>)
>     at ../glib/gmain.c:4175
> #5  0x00007f51ca1b6ea2 in g_main_loop_run (loop=0x555b2f21fbf0)
>     at ../glib/gmain.c:4373
> #6  0x0000555b2e2d69a1 in main (argc=<optimized out>, argv=<optimized out>)
>     at ../src/core/main.c:509
> (gdb) l
> 601     ipv6_dad_timeout(gpointer user_data)
> 602     {
> 603         NMDhcpClient        *self = user_data;
> 604         NMDhcpClientPrivate *priv = NM_DHCP_CLIENT_GET_PRIVATE(self);
> 605
> 606         nm_clear_g_source_inst(&priv->v6.dad_timeout_source);
> 607
> 608         _emit_notify(
> 609             self,
> 610             &((NMDhcpClientNotifyData){
> (gdb) p priv
> $1 = (NMDhcpClientPrivate *) 0x0

What about `p self` when NetworkManager crashes? Is it also null?
Comment 9 Andreas Schwab 2023-10-22 07:35:30 UTC 
Of course not, otherwise NM_DHCP_CLIENT_GET_PRIVATE(self) would have crashed.
Comment 10 Andreas Schwab 2023-11-18 15:02:37 UTC 
Any news?  Still crashing and crashing.
Comment 11 Jonathan Kang 2023-11-22 06:44:40 UTC 
I was occupied by other things in the past weeks. I'll take a further look at this issue this week.

Can you provide another NetworkManager journal and note exactly when the crash happened? I didn't find much useful information in the previous one.

Thanks.
Comment 12 Andreas Schwab 2023-11-22 18:25:31 UTC 
The crash ends the log, of course.
Comment 13 Andreas Schwab 2023-11-26 08:50:57 UTC 
#0  g_source_destroy (source=0x373a37383a31393a) at ../glib/gmain.c:1386
#1  0x000056253edcda61 in nm_g_source_destroy_and_unref ()
    at ../src/libnm-glib-aux/nm-shared-utils.h:1699
#2  ipv6_dad_timeout (user_data=user_data@entry=0x562540a40df0)
    at ../src/core/dhcp/nm-dhcp-client.c:607
#3  0x00007f4140f9637d in g_timeout_dispatch (source=0x562540bead50, 
    callback=0x56253edcda20 <ipv6_dad_timeout>, user_data=0x562540a40df0)
    at ../glib/gmain.c:4933
#4  0x00007f4140f9582b in g_main_dispatch (context=0x562540a23890)
    at ../glib/gmain.c:3381
#5  g_main_context_dispatch (context=context@entry=0x562540a23890)
    at ../glib/gmain.c:4099
#6  0x00007f4140f95bd0 in g_main_context_iterate (context=0x562540a23890, 
    block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>)
    at ../glib/gmain.c:4175
#7  0x00007f4140f95ea2 in g_main_loop_run (loop=0x562540a21c70)
    at ../glib/gmain.c:4373
#8  0x000056253edc39a1 in main (argc=<optimized out>, argv=<optimized out>)
    at ../src/core/main.c:509
Comment 14 Jonathan Kang 2023-11-27 09:03:36 UTC 
(In reply to Andreas Schwab from comment #13)
> #0  g_source_destroy (source=0x373a37383a31393a) at ../glib/gmain.c:1386
> #1  0x000056253edcda61 in nm_g_source_destroy_and_unref ()
>     at ../src/libnm-glib-aux/nm-shared-utils.h:1699
> #2  ipv6_dad_timeout (user_data=user_data@entry=0x562540a40df0)
>     at ../src/core/dhcp/nm-dhcp-client.c:607
> #3  0x00007f4140f9637d in g_timeout_dispatch (source=0x562540bead50, 
>     callback=0x56253edcda20 <ipv6_dad_timeout>, user_data=0x562540a40df0)
>     at ../glib/gmain.c:4933
> #4  0x00007f4140f9582b in g_main_dispatch (context=0x562540a23890)
>     at ../glib/gmain.c:3381
> #5  g_main_context_dispatch (context=context@entry=0x562540a23890)
>     at ../glib/gmain.c:4099
> #6  0x00007f4140f95bd0 in g_main_context_iterate (context=0x562540a23890, 
>     block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>)
>     at ../glib/gmain.c:4175
> #7  0x00007f4140f95ea2 in g_main_loop_run (loop=0x562540a21c70)
>     at ../glib/gmain.c:4373
> #8  0x000056253edc39a1 in main (argc=<optimized out>, argv=<optimized out>)
>     at ../src/core/main.c:509

Can you share another NetworkManager journal when it crashes again?

Thanks
Comment 15 Andreas Schwab 2024-03-03 10:19:24 UTC 
#0  0x0000562208a33a45 in ipv6_dad_timeout (
    user_data=user_data@entry=0x562208e1ef20)
    at ../src/core/dhcp/nm-dhcp-client.c:606
#1  0x00007fdb5fc7237d in g_timeout_dispatch (source=0x562208f5fec0, 
    callback=0x562208a33a20 <ipv6_dad_timeout>, user_data=0x562208e1ef20)
    at ../glib/gmain.c:4933
#2  0x00007fdb5fc7182b in g_main_dispatch (context=0x562208e01890)
    at ../glib/gmain.c:3381
#3  g_main_context_dispatch (context=context@entry=0x562208e01890)
    at ../glib/gmain.c:4099
#4  0x00007fdb5fc71bd0 in g_main_context_iterate (context=0x562208e01890, 
    block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>)
    at ../glib/gmain.c:4175
#5  0x00007fdb5fc71ea2 in g_main_loop_run (loop=0x562208dffc70)
    at ../glib/gmain.c:4373
#6  0x0000562208a299a1 in main (argc=<optimized out>, argv=<optimized out>)
    at ../src/core/main.c:509
Comment 16 Andreas Schwab 2024-03-03 10:23:31 UTC 
$ coredumpctl list | tail -n 20
Sun 2024-03-03 10:57:31 CET   1138    0   0 SIGSEGV present  /usr/sbin/NetworkManager   791.7K
Sun 2024-03-03 10:59:44 CET   6859    0   0 SIGSEGV present  /usr/sbin/NetworkManager   815.7K
Sun 2024-03-03 11:00:55 CET   7649    0   0 SIGSEGV present  /usr/sbin/NetworkManager   822.0K
Sun 2024-03-03 11:01:30 CET   8470    0   0 SIGSEGV present  /usr/sbin/NetworkManager   837.9K
Sun 2024-03-03 11:03:24 CET   9288    0   0 SIGSEGV present  /usr/sbin/NetworkManager   851.2K
Sun 2024-03-03 11:04:30 CET  10548    0   0 SIGSEGV present  /usr/sbin/NetworkManager   804.0K
Sun 2024-03-03 11:05:09 CET  11443    0   0 SIGSEGV present  /usr/sbin/NetworkManager   807.8K
Sun 2024-03-03 11:05:43 CET  12483    0   0 SIGSEGV present  /usr/sbin/NetworkManager   803.5K
Sun 2024-03-03 11:06:18 CET  12968    0   0 SIGSEGV present  /usr/sbin/NetworkManager   783.5K
Sun 2024-03-03 11:07:36 CET  13463    0   0 SIGSEGV present  /usr/sbin/NetworkManager   786.8K
Sun 2024-03-03 11:08:11 CET  14098    0   0 SIGSEGV present  /usr/sbin/NetworkManager   789.0K
Sun 2024-03-03 11:08:46 CET  14595    0   0 SIGSEGV present  /usr/sbin/NetworkManager   789.1K
Sun 2024-03-03 11:10:21 CET  15076    0   0 SIGSEGV present  /usr/sbin/NetworkManager   813.4K
Sun 2024-03-03 11:10:56 CET  15775    0   0 SIGSEGV present  /usr/sbin/NetworkManager   805.3K
Sun 2024-03-03 11:12:43 CET  16541    0   0 SIGSEGV present  /usr/sbin/NetworkManager   784.6K
Sun 2024-03-03 11:13:17 CET  17534    0   0 SIGSEGV present  /usr/sbin/NetworkManager   797.0K
Sun 2024-03-03 11:14:23 CET  18012    0   0 SIGSEGV present  /usr/sbin/NetworkManager   820.9K
Sun 2024-03-03 11:15:50 CET  19235    0   0 SIGSEGV present  /usr/sbin/NetworkManager   825.4K
Sun 2024-03-03 11:16:58 CET  20154    0   0 SIGSEGV present  /usr/sbin/NetworkManager   826.9K
Sun 2024-03-03 11:19:33 CET  20720    0   0 SIGSEGV present  /usr/sbin/NetworkManager   793.8K
Comment 17 Jonathan Kang 2024-03-22 02:02:26 UTC 
Hi Andreas

Sorry for lacking of response here. I'm occupied with other work items atm. You can report this at upstream gitlab so that they can also take a look at this issue.

* https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/issues