Bugzilla – Bug 1188949
dhclient unable to obtain an IP address
Last modified: 2023-06-15 02:02:05 UTC
Created attachment 851456 [details] NetworkManager logs, inxi-Fxz output, video and photo with error demonstration After installation, the wired connection works correctly. The next day, it stops connecting. There is a network connection that lasts indefinitely. Creating new connections does not lead to any result, as does trying to reconnect to an existing connection. It only helps to roll back the system to the backup created immediately after installation. The bug manifests itself on all DE and the next day after installation. During installation, the system configures the wired connection itself and the installation takes place without any errors. The problem manifests itself in openSUSE Leap and Tumbleweed, the wired connection works correctly on other distributions. I tried to install the r8168 driver, and add r8169 to the blacklist, but this does not help. Installing a third-party custom kernel also does not fix the problem. IPv4, IPv6, DNS and the default route disappear from the connection settings. I looked at the Network Manager configs, but there is nothing suspicious there, because of which the problem could manifest itself.
The logs indicate that it's trying to get a dynamic IP via dhclient but didn't get one till you manually turn off the connection. Can you confirm DHCP is properly configured? BTW, add the following to /etc/NetworkManager/NetworkManager.conf > [logging] > level=trace restart NetworkManager and reproduce this again. This turns of more verbose logging in NetworkManager.
Created attachment 851677 [details] journalctl -b -u NetworkManager Here is the NetworkManager log for today. It was not possible to connect the first time, but it turned out the second time. Sometimes it is possible to randomly establish a connection. The DHCP must be configured correctly. Before that, I worked on Linux Mint for a year, I never had such a problem.
(In reply to Vladislav Milenin from comment #2) > Created attachment 851677 [details] > journalctl -b -u NetworkManager > > Here is the NetworkManager log for today. It was not possible to connect the > first time, but it turned out the second time. Sometimes it is possible to > randomly establish a connection. The DHCP must be configured correctly. > Before that, I worked on Linux Mint for a year, I never had such a problem. Do you mind uploading the content of "journalctl -b" instead of journal only from NetworkManager? You mentioned creating a new connection doesn't help either. Could you show the output of "nmcli connection show $newly-created-connection-name" as well? Thanks.
Created attachment 851769 [details] Logs of journalctl and output of nmcli Yes, creating a new connection does not help, as does reconnecting to an existing one.
(In reply to Vladislav Milenin from comment #4) > Created attachment 851769 [details] > Logs of journalctl and output of nmcli > > Yes, creating a new connection does not help, as does reconnecting to an > existing one. It looks like the system never got an ip address through dhcp. But there is no detailed log messages in the default NetworkManager logs. I've enabled verbose log messages in this build[0] of NetworkManager. Can you install this rpm and attach logs here? *[0] https://build.opensuse.org/package/show/home:JonathanKang:branches:GNOME:Factory/NetworkManager
Created attachment 851959 [details] NetworkManager logs from a third-party repository Today I managed to connect to the Internet on the second attempt. Sometimes this happens, but I have not found any pattern. This happens purely by chance.
(In reply to Vladislav Milenin from comment #6) > Created attachment 851959 [details] > NetworkManager logs from a third-party repository > > Today I managed to connect to the Internet on the second attempt. Sometimes > this happens, but I have not found any pattern. This happens purely by > chance. It looks like you didn't install NetworkManager from the provided repo in comment#5. BTW, as a workaround, you may try to change > dhcp=dhclient to > dhcp=internal in /etc/NetworkManager/NetworkManager.conf.
Created attachment 852086 [details] journalctl -b -u NetworkManager from a third-party repository Yes, I noticed that I installed this package incorrectly. I installed it for the second time, but this time correctly. Even in the config, NetworkManager replaced dhcp=dhclient to dhcp=internal. I noticed a very strange behavior of the network. The network was connected to the Internet normally for two days, but this morning it refused to connect again. I had to use the phone as a USB modem. In the afternoon, I turned it on again and the network was already connected normally. The strange thing is that I noticed that if you connect the Internet via your phone as a USB modem, then after disconnecting it from the computer, the network sometimes connects correctly for some time.
(In reply to Vladislav Milenin from comment #8) > Created attachment 852086 [details] > journalctl -b -u NetworkManager from a third-party repository > > Yes, I noticed that I installed this package incorrectly. I installed it for > the second time, but this time correctly. Even in the config, NetworkManager > replaced dhcp=dhclient to dhcp=internal. I noticed a very strange behavior > of the network. The network was connected to the Internet normally for two > days, but this morning it refused to connect again. I had to use the phone > as a USB modem. In the afternoon, I turned it on again and the network was > already connected normally. The strange thing is that I noticed that if you > connect the Internet via your phone as a USB modem, then after disconnecting > it from the computer, the network sometimes connects correctly for some time. The logs indicates that during the phase of DHCPDISCOVER, the ethernet link was disconnected somehow and your system cannot get DHCPOFFER from the dhcp server. But without the whole "journalctl -b" for the system, I'm unable to tell exactly why the link was disconnected and no carrier was available for the ethernet interface. Could you please attach the output of "journalctl -b" here?
Created attachment 852115 [details] Logs of journalctl There have been no connection failures since the last time
Created attachment 852201 [details] Logs of dhcp=dhclient After replacing dhcp=dhclient to dhcp=internal, the Internet was connected correctly for these few days. But it was worth changing these values back, as the same problem reappeared. In a non-working state, I took the logs again and returned them back to dhcp=internal, since it works better with it.
(In reply to Vladislav Milenin from comment #11) > Created attachment 852201 [details] > Logs of dhcp=dhclient > > After replacing dhcp=dhclient to dhcp=internal, the Internet was connected > correctly for these few days. But it was worth changing these values back, > as the same problem reappeared. In a non-working state, I took the logs > again and returned them back to dhcp=internal, since it works better with it. Can you install NetworkManager from the repo mentioned in comment#5 and provide logs again? More verbose messages are needed to debug dhclient. Thanks.
Created attachment 852275 [details] NetworkManager logs I am attaching the logs and a screen from which you can see that NetworkManager is installed from the repository you need, to which it changed, after installing from the .ymp package from comment#5.
(In reply to Jonathan Kang from comment #12) > (In reply to Vladislav Milenin from comment #11) > > Created attachment 852201 [details] > > Logs of dhcp=dhclient > > > > After replacing dhcp=dhclient to dhcp=internal, the Internet was connected > > correctly for these few days. But it was worth changing these values back, > > as the same problem reappeared. In a non-working state, I took the logs > > again and returned them back to dhcp=internal, since it works better with it. > > Can you install NetworkManager from the repo mentioned in comment#5 and > provide > logs again? More verbose messages are needed to debug dhclient. > > Thanks. I conducted additional testing. It seems that changing the parameter in the NM config to dhcp=internal solves the problem, because after that the Internet began to connect normally. But I still did not understand why such an error is obtained with the parameter dhcp=dhclient. You asked to install NM from comment#5, which I did. YaST notified about the correct installation, then restarted the computer and took the logs. But you write that NM is still installed from the standard openSUSE repositories. I checked again and left a screenshot of YaST in comment#13, where it is clear that NM is installed from the GNOME repository:Factory, which was added after installing the package.
(In reply to Vladislav Milenin from comment #14) > (In reply to Jonathan Kang from comment #12) > > (In reply to Vladislav Milenin from comment #11) > > > Created attachment 852201 [details] > > > Logs of dhcp=dhclient > > > > > > After replacing dhcp=dhclient to dhcp=internal, the Internet was connected > > > correctly for these few days. But it was worth changing these values back, > > > as the same problem reappeared. In a non-working state, I took the logs > > > again and returned them back to dhcp=internal, since it works better with it. > > > > Can you install NetworkManager from the repo mentioned in comment#5 and > > provide > > logs again? More verbose messages are needed to debug dhclient. > > > > Thanks. > > I conducted additional testing. It seems that changing the parameter in the > NM config to dhcp=internal solves the problem, because after that the > Internet began to connect normally. I suspect that it's a dhclient issue, so I enabled verbose in the package provided in comment#5. > But I still did not understand why such > an error is obtained with the parameter dhcp=dhclient. You asked to install > NM from comment#5, which I did. YaST notified about the correct > installation, then restarted the computer and took the logs. But you write > that NM is still installed from the standard openSUSE repositories. I > checked again and left a screenshot of YaST in comment#13, where it is clear > that NM is installed from the GNOME repository:Factory, which was added > after installing the package. You should be installing NetworkManager from home:JonathanKang:braches:GNOME:Factory instead of GNOME:Factory.
Created attachment 852384 [details] Logs NM of JonathanKang braches GNOME Factory I changed the NM repository to home:Jonathan Kang:branches:GNOME:Factory and took the logs again.
(In reply to Vladislav Milenin from comment #16) > Created attachment 852384 [details] > Logs NM of JonathanKang braches GNOME Factory > > I changed the NM repository to home:Jonathan Kang:branches:GNOME:Factory and > took the logs again. dhclient never receives an DHCPACK after sending DHCPREQUEST message. But I have no idea exactly why. I guess you're using a router, so it won't be possible to get dhcp related logs from the dhcp server side.
(In reply to Jonathan Kang from comment #17) > (In reply to Vladislav Milenin from comment #16) > > Created attachment 852384 [details] > > Logs NM of JonathanKang braches GNOME Factory > > > > I changed the NM repository to home:Jonathan Kang:branches:GNOME:Factory and > > took the logs again. > > dhclient never receives an DHCPACK after sending DHCPREQUEST message. But I > have > no idea exactly why. I guess you're using a router, so it won't be possible > to > get dhcp related logs from the dhcp server side. Yes, I use a Zyxel keenetic lite III router. The router was configured by the master once, during installation. For all the time of operation, not a single device had any connection errors via a wired connection and Wi-Fi.
Hi Reinhard I guess you're the maintainer of dhclient. In this bug, NetworkManager spawned dhclient to get an ip from dhcp server. But after the client send DHCPREQUEST, it never got an DHCPACK reply. Do you have any idea about what could go wrong here? > сен 08 18:26:56 localhost dhclient[1047]: Internet Systems Consortium DHCP Client 4.4.2-P1 > сен 08 18:26:56 localhost dhclient[1047]: Copyright 2004-2021 Internet Systems Consortium. > сен 08 18:26:56 localhost dhclient[1047]: All rights reserved. > сен 08 18:26:56 localhost dhclient[1047]: For info, please visit https://www.isc.org/software/dhcp/ > сен 08 18:26:56 localhost dhclient[1047]: > сен 08 18:26:56 localhost dhclient[1047]: Listening on LPF/enp2s0/70:8b:cd:9e:5b:b2 > сен 08 18:26:56 localhost dhclient[1047]: Sending on LPF/enp2s0/70:8b:cd:9e:5b:b2 > сен 08 18:26:56 localhost dhclient[1047]: Sending on Socket/fallback > сен 08 18:26:56 localhost dhclient[1047]: DHCPDISCOVER on enp2s0 to 255.255.255.255 port 67 interval 5 (xid=0x1b7bdecb) > сен 08 18:26:56 localhost dhclient[1047]: DHCPOFFER of 192.168.1.43 from 192.168.1.1 > сен 08 18:27:01 localhost dhclient[1047]: DHCPREQUEST for 192.168.1.43 on enp2s0 to 255.255.255.255 port 67 (xid=0x1b7bdecb) > сен 08 18:27:05 localhost dhclient[1047]: DHCPREQUEST for 192.168.1.43 on enp2s0 to 255.255.255.255 port 67 (xid=0x1b7bdecb) > сен 08 18:27:11 localhost dhclient[1047]: DHCPREQUEST for 192.168.1.43 on enp2s0 to 255.255.255.255 port 67 (xid=0x1b7bdecb) Complete version of these logs are in comment#16.
(In reply to Jonathan Kang from comment #19) > I guess you're the maintainer of dhclient. yes, I am the maintainer of the dhcp package, but not a deep expert in DHCP. > Do you have any idea about what could go wrong here? I discussed this with Marius, but we couldn't find a possible reason within dhclient that would explain this. To narrow this further down we want to see if the missing DHCPACK packages are not being sent by the router, being filtered by the kernel, or being silently ignored by dhclient, because it thinks they are invalid. For that, please disable your firewall (if it is active) and change some settings: # sysctl -w net.ipv4.conf.all.log_martians=1 # sysctl -w sysctl -w net.ipv4.conf.all.rp_filter=0 Then record all DHCP packages over at least one occurance of dhclient sending a request and seemingly getting no ack: # tcpdump -W /tmp/dhcp4.pcapng -s 65535 -Uvnfi any 'port 67 or port 68' Interrupt tcpdump with Ctrl-C when done and upload /tmp/dhcp4.pcapng as an attachment.
(In reply to Reinhard Max from comment #20) > (In reply to Jonathan Kang from comment #19) > > > I guess you're the maintainer of dhclient. > > yes, I am the maintainer of the dhcp package, but not a deep expert in DHCP. > > > Do you have any idea about what could go wrong here? > > I discussed this with Marius, but we couldn't find a possible reason within > dhclient that would explain this. To narrow this further down we want to see > if the missing DHCPACK packages are not being sent by the router, being > filtered by the kernel, or being silently ignored by dhclient, because it > thinks they are invalid. > > For that, please disable your firewall (if it is active) and change some > settings: > > # sysctl -w net.ipv4.conf.all.log_martians=1 > # sysctl -w sysctl -w net.ipv4.conf.all.rp_filter=0 > > Then record all DHCP packages over at least one occurance of dhclient > sending a request and seemingly getting no ack: > > # tcpdump -W /tmp/dhcp4.pcapng -s 65535 -Uvnfi any 'port 67 or port 68' > > Interrupt tcpdump with Ctrl-C when done and upload /tmp/dhcp4.pcapng as an > attachment. I have a few questions. I need to use NetworkManager from the openSUSE repositories or from the home repository:Jonathan Kang:branches:GNOME:Factory. If it is from the openSUSE repositories, is it in working condition or when the Internet stops connecting?
(In reply to Vladislav Milenin from comment #21) > (In reply to Reinhard Max from comment #20) > > (In reply to Jonathan Kang from comment #19) > > > > > I guess you're the maintainer of dhclient. > > > > yes, I am the maintainer of the dhcp package, but not a deep expert in DHCP. > > > > > Do you have any idea about what could go wrong here? > > > > I discussed this with Marius, but we couldn't find a possible reason within > > dhclient that would explain this. To narrow this further down we want to see > > if the missing DHCPACK packages are not being sent by the router, being > > filtered by the kernel, or being silently ignored by dhclient, because it > > thinks they are invalid. > > > > For that, please disable your firewall (if it is active) and change some > > settings: > > > > # sysctl -w net.ipv4.conf.all.log_martians=1 > > # sysctl -w sysctl -w net.ipv4.conf.all.rp_filter=0 > > > > Then record all DHCP packages over at least one occurance of dhclient > > sending a request and seemingly getting no ack: > > > > # tcpdump -W /tmp/dhcp4.pcapng -s 65535 -Uvnfi any 'port 67 or port 68' > > > > Interrupt tcpdump with Ctrl-C when done and upload /tmp/dhcp4.pcapng as an > > attachment. > > I have a few questions. I need to use NetworkManager from the openSUSE > repositories or from the home repository:Jonathan > Kang:branches:GNOME:Factory. Either is okay. > If it is from the openSUSE repositories, is it > in working condition or when the Internet stops connecting? This hasn't be fixed. So it's not in proper working condition. We want a bit more information from your system for further debug this issue. Thanks.
(In reply to Vladislav Milenin from comment #21) > If it is from the openSUSE repositories, is it > in working condition or when the Internet stops connecting? We need at least a package dump taken while the problem shows on your system, but if you can also take a sample while things are working, that would be even better.
Created attachment 852512 [details] Error command For some reason, the last command from comment#20 does not work. I attached a screen with an error.
Whoops, sorry, I made some typos and copy&paste errors when posting the commands. Here we go again: # sysctl -w net.ipv4.conf.all.log_martians=1 # sysctl -w net.ipv4.conf.all.rp_filter=0 # tcpdump -w /tmp/dhcp4.pcapng -s 65535 -Uvnfi any 'port 67 or port 68'
Created attachment 852544 [details] dhcp4.pcapng file This time everything turned out correctly.
Created attachment 852545 [details] dhcp4.pcapng file without firewall I'm very sorry, I forgot to disable the firewall. Here is the new file where I disabled the firewall.
(In reply to Vladislav Milenin from comment #27) > Created attachment 852545 [details] > dhcp4.pcapng file without firewall > > I'm very sorry, I forgot to disable the firewall. Here is the new file where > I disabled the firewall. There two are empty file without any package inside. You need to reproduce this issue with tcpdump command running.
Created attachment 852630 [details] dhcp4.pcapng file This time it should work out correctly.
Created attachment 852675 [details] dhcp4.pcapng correct Another dhcp 4. pcapng file, this time not empty.
Did the last dhcp 4.pcapng file that I threw off give any information about the problem?
It just shows that the kernel does not receive any ACK packages at this point in time, and so the DHCP client keeps retrying and retrying. It would be good if you could take another capture of the working situation, taken during a network restart to enforce DHCP re-negotiation.
Created attachment 852755 [details] dhcp4 reloaded It should have turned out correctly.
Is there any result from the last attached file?
It contains no ACK packets either, so there is nothing new to learn from it.
(In reply to Reinhard Max from comment #35) > It contains no ACK packets either, so there is nothing new to learn from it. It turns out that as a solution to the problem it is worth using the dhcp = internal parameter from comment#7, since it was not possible to establish the reason for the incorrect operation of the dhcp = dhclient parameter?
(In reply to Vladislav Milenin from comment #36) > (In reply to Reinhard Max from comment #35) > > It contains no ACK packets either, so there is nothing new to learn from it. > > It turns out that as a solution to the problem it is worth using the dhcp = > internal parameter from comment#7, since it was not possible to establish > the reason for the incorrect operation of the dhcp = dhclient parameter? Yes, you can use "dhcp=internal" as a solution/workaround.
(In reply to Jonathan Kang from comment #37) > (In reply to Vladislav Milenin from comment #36) > > (In reply to Reinhard Max from comment #35) > > > It contains no ACK packets either, so there is nothing new to learn from it. > > > > It turns out that as a solution to the problem it is worth using the dhcp = > > internal parameter from comment#7, since it was not possible to establish > > the reason for the incorrect operation of the dhcp = dhclient parameter? > > Yes, you can use "dhcp=internal" as a solution/workaround. What then to do with an open bug? I think it shouldn't be closed if new details or ideas appear in the future.
(In reply to Vladislav Milenin from comment #38) > > What then to do with an open bug? I think it shouldn't be closed if new > details or ideas appear in the future. This should be kept open until further progress is made and it's fixed.
@Vladislav Can you try to disable the wired connection and then run "dhclient -v -d eth0"? Please report back whether you can get an IP address.
Created attachment 854023 [details] dhclient log When executing the command "dhclient -v -d eth0" nothing happens and "Device "eth0" does not exist." is output. My network is called enp2s0, as the system automatically named it during installation. In the command, I changed "eth0" to "enp2s0" and it was executed correctly.
(In reply to Vladislav Milenin from comment #41) > Created attachment 854023 [details] > dhclient log > > When executing the command "dhclient -v -d eth0" nothing happens and "Device > "eth0" does not exist." is output. My network is called enp2s0, as the > system automatically named it during installation. In the command, I changed > "eth0" to "enp2s0" and it was executed correctly. Yep. Use your ethernet interface name instead. The log shows that dhclient still cannot get an reply from DHCP server. It didn't even get an DHCPOFFER this time.
(In reply to Jonathan Kang from comment #42) > (In reply to Vladislav Milenin from comment #41) > > Created attachment 854023 [details] > > dhclient log > > > > When executing the command "dhclient -v -d eth0" nothing happens and "Device > > "eth0" does not exist." is output. My network is called enp2s0, as the > > system automatically named it during installation. In the command, I changed > > "eth0" to "enp2s0" and it was executed correctly. > > Yep. Use your ethernet interface name instead. > > The log shows that dhclient still cannot get an reply from DHCP server. It > didn't > even get an DHCPOFFER this time. I've run out of ideas what it could be. Most likely, this is a unique bug on my configuration, which cannot be reproduced by anyone else. At the moment, as a solution, I use the dhcp=internal parameter from comment#7 and it works fine all the time.
*** Bug 1188736 has been marked as a duplicate of this bug. ***
(In reply to Vladislav Milenin from comment #43) > (In reply to Jonathan Kang from comment #42) > > (In reply to Vladislav Milenin from comment #41) > > > Created attachment 854023 [details] > > > dhclient log > > > > > > When executing the command "dhclient -v -d eth0" nothing happens and "Device > > > "eth0" does not exist." is output. My network is called enp2s0, as the > > > system automatically named it during installation. In the command, I changed > > > "eth0" to "enp2s0" and it was executed correctly. > > > > Yep. Use your ethernet interface name instead. > > > > The log shows that dhclient still cannot get an reply from DHCP server. It > > didn't > > even get an DHCPOFFER this time. > > I've run out of ideas what it could be. Most likely, this is a unique bug on > my configuration, which cannot be reproduced by anyone else. At the moment, > as a solution, I use the dhcp=internal parameter from comment#7 and it works > fine all the time. I have a brand new mini PC ASUS ExpertCenter PN52 and I have the same problem. The network disconnects every few days. I have not noticed the same error in another mini PC (Intel® NUC Kit NUC8i7BEH). Both computers have the same software Tumbleweed 20221202.
I have a brand new mini PC ASUS ExpertCenter PN52 and I have the same problem. The network disconnects every few days. I have not noticed the same error in another mini PC (Intel® NUC Kit NUC8i7BEH). Both computers have the same software Tumbleweed 20221202.
I have spent my time researching this problem, even though I am only a 65 year old retired female engineer. I noticed that journalctl had a lot of 'localhost.localdomain nscd[949]: 949 checking for monitored file `/etc/services': No such file or directory' log (see No such file or directory.txt). I copied the services file: cp /usr/etc/services /etc/ and it seems that the network connection stays on now (see the end of No such file or directory.txt) The failure is on in log file: failure.txt Startup of PN52 in log file: startup.txt p.s. I'll let you know if/when the network connection fails again.
Created attachment 863376 [details] Journalctl log when network is down
Created attachment 863377 [details] journalctl computer startup log after network failure
Created attachment 863378 [details] No such file or directory.txt
Created attachment 863379 [details] information about my computer
The error came again. For a few days the network connection worked ok. The connection can be revived at least this way: 1. close the computer and disconnect the LAN cable. 2. start the computer 3. when it is running, then connect the LAN cable Operating System: openSUSE Tumbleweed 20221211 KDE Plasma Version: 5.26.4 KDE Frameworks Version: 5.100.0 Qt Version: 5.15.7 Kernel Version: 6.0.12-1-default (64-bit) Graphics Platform: X11 Processors: 16 x AMD Ryzen 9 5900HX with Radeon Graphics Memory: 30.8 GiB of RAM Graphics Processor: AMD Radeon Graphics Manufacturer: ASUSTeK COMPUTER INC. Product Name: MINIPC PN52 p.s. does anyone read my writings and logs, or am I collecting information in vain?
Created attachment 863480 [details] journalctl log when the error occurred
Created attachment 863481 [details] journalctl log after reboot
Raija, thanks for your efforts! The logs you provided are appreciated. But your issue seems to have a different reason than Vladislav's, although the symptoms might look similar at first sight. Both systems have network cards that are run by the r8169 driver, but in your case the kernel reports an error and stack trace about the driver when the network goes down (starting with "NETDEV WATCHDOG: enp2s0 (r8169): transmit queue 0 timed out"), whereas in Vladislav's case no kernel errors happen, but DHCP-ACK packets are missing. I'll have another look at this next week and then maybe open a separate bug report for your issue. BTW, r8169 has quite a history of being unstable in various ways, I also had huge problems with it on a rented server a few years ago where I found no working solution and ended up using an Intel network card instead. Of course this doesn't help if the network chip is on-board and the board has no slot for an add-on network card.
On New Year's Eve, I bought a new SSD drive and reinstalled the system. It has been working fine for more than two weeks and the bug has not appeared again.
FYI, I disabled the Powerline-Ethernet-Kit, and the problem has not occurred anymore. So apparently the fault was in the cooperation between that device and the ASUS PN52? https://www.amazon.com/Zyxel-PLA-401-Powerline-Ethernet-Kit/dp/B008J4ART8
NetworkManager in Tumbleweed defaults to use its internal dhcp client instead of dhclient for some time. This issue is dhclient specific, and it's difficult to reproduce as well. So closing this as RESOLVED WORKSFORME. Feel free to re-open if this can still be reproduced in your system.