Bugzilla – Bug 1224327
"Client hit max requests limit 1" during live migrations
Last modified: 2024-07-12 16:30:50 UTC
Hi, on our KVM cluster we observe lots of the following in the virtlockd and virtlogd logs during live migrations: "Client hit max requests limit 1. This may result in keep-alive timeouts. Consider tuning the max_client_requests server parameter" I tried setting "max_client_requests = 256" in virtlockd.conf, virtlogd.conf, and the main libvirtd.conf, but the message remains. I don't quite understand where the limit of 1 is coming from. Any ideas? Currently we have the following installed, but this already happens since quite some time, I only now got to report it: ``` falkor22 (Hypervisor):~ # grep PRETTY /etc/os-release PRETTY_NAME="openSUSE Leap 15.5" falkor22 (Hypervisor):~ # rpm -qa *libvirt* python3-libvirt-python-9.0.0-150500.1.4.x86_64 libvirt-libs-9.0.0-150500.6.20.1.x86_64 libvirt-daemon-driver-storage-core-9.0.0-150500.6.20.1.x86_64 libvirt-client-9.0.0-150500.6.20.1.x86_64 system-group-libvirt-20170617-150400.24.2.1.noarch libvirt-daemon-9.0.0-150500.6.20.1.x86_64 libvirt-daemon-driver-qemu-9.0.0-150500.6.20.1.x86_64 libvirt-daemon-driver-storage-mpath-9.0.0-150500.6.20.1.x86_64 libvirt-daemon-config-network-9.0.0-150500.6.20.1.x86_64 libvirt-daemon-driver-network-9.0.0-150500.6.20.1.x86_64 libvirt-daemon-driver-storage-disk-9.0.0-150500.6.20.1.x86_64 ``` Best, Georg
Forgot to add, our migrations are triggered through Pacemaker which calls the following (just one example): ``` virsh --connect=qemu:///system --quiet migrate --live netbox1.infra.opensuse.org qemu+tcp://falkor21.infra.opensuse.org/system ```
Probably it's a libvirt issue. Below repo contains a potential fix. Could you please update your libvirt* via it to see whether the issue is fixed? https://download.suse.de/ibs/home:/lin_ma:/branches:/SUSE:/SLE-15-SP5:/Update/standard/
Hi Lin, thanks for getting back. Could you please provide your updated packages through OBS?
(In reply to Georg Pfuetzenreuter from comment #3) > Hi Lin, > > thanks for getting back. Could you please provide your updated packages > through OBS? Sure. https://download.opensuse.org/repositories/home:/lin_ma:/branches:/SUSE:/SLE-15-SP5:/Update/pool-leap-15.5/
Thanks! I will test on the weekend and let you know.
Hi, your package seems to work fine. Can we get it into a maintenance update? Also, do I still need the `max_client_requests` setting in the three configuration files or is it superfluous?
(In reply to Georg Pfuetzenreuter from comment #6) > Hi, > > your package seems to work fine. Glad to know the issue can be fixed by the package. > Can we get it into a maintenance update? Sure. @Jim, The issue can be fixed by libvirt upstream commit b3f8e072. Could you please help to backport it into SLE15SP5 and leap 15.5? > Also, do I still need the `max_client_requests` setting in the three > configuration files or is it superfluous? It is unnecessary. You don't need the `max_client_requests` setting, Please comment it like before.
(In reply to Lin Ma from comment #7) Hi Lin, Thanks for helping with this bug while I was on vacation! > @Jim, The issue can be fixed by libvirt upstream commit b3f8e072. > Could you please help to backport it into SLE15SP5 and leap 15.5? I've added b3f8e072 to the SLE15 SP5 libvirt package and submitted it for maintenance.
SUSE-RU-2024:1933-1: An update that has two fixes can now be installed. Category: recommended (moderate) Bug References: 1220714, 1224327 Maintenance Incident: [SUSE:Maintenance:34092](https://smelt.suse.de/incident/34092/) Sources used: openSUSE Leap 15.5 (src): libvirt-9.0.0-150500.6.23.1 SUSE Linux Enterprise Micro 5.5 (src): libvirt-9.0.0-150500.6.23.1 Basesystem Module 15-SP5 (src): libvirt-9.0.0-150500.6.23.1 Server Applications Module 15-SP5 (src): libvirt-9.0.0-150500.6.23.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
SUSE-RU-2024:1933-2: An update that has two fixes can now be installed. Category: recommended (moderate) Bug References: 1220714, 1224327 Maintenance Incident: [SUSE:Maintenance:34092](https://smelt.suse.de/incident/34092/) Sources used: SUSE Linux Enterprise Micro 5.5 (src): libvirt-9.0.0-150500.6.23.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.