Bugzilla – Bug 1216878
server:monitoring/monitoring-plugins: check_users reporting negative values
Last modified: 2024-02-13 09:47:31 UTC
Started happening today on my openSUSE MicroOS, I think that after the last update. > # who|wc -l > 2 But calling: > /usr/lib/nagios/plugins/check_users -w 5 -c 10 Should return OK, but instead: > USERS CRITICAL - -13 users currently logged in |users=-13;5;15;0 It returns a negative number. I see monitoring-plugins-users got updated: > 2023-11-03 01:49:19|install|monitoring-plugins-users|2.3.3-1.1|x86_64||repo-oss|8c063521869d4ad1ee1c2e409fd06ffa09cbe6f86ee3cf732a6213895796eba9c396a06ec53efac2b315d0920490e8f5bbd508da0396327432e800b5d0624ccb| Checking the changelog, I think it's an update to fix https://bugzilla.opensuse.org/show_bug.cgi?id=1216148 at the source package monitorning-plugins, but I can't figure out how that should affect check_users. >* Mon Oct 30 2023 ro@suse.de >- renamed patches > monitoring-plugins-wrong_return_in_check_swap.patch > to monitoring-plugins-2.3.3-wrong_percent_in_check_swap.patch > monitoring-plugins.check_snmp.arrayaddress.patch > to monitoring-plugins-2.3.3-check_snmp.arrayaddress.patch > monitoring-plugins-2.3-check_ntp_perf_absolute.patch > to monitoring-plugins-2.3.3-check_ntp_perf_absolute.patch > >* Mon Oct 16 2023 Thorsten Kukuk <kukuk@suse.com> >- Add buildrequires for coreutils-systemd, as the configure script > checks for uptime [bsc#1216148] I wonder if it's another missing dependency that is not in the list of buildrequires. Are we maybe cleaning up the list of packages preinstalled on buildes for Factory? Reading https://github.com/nagios-plugins/nagios-plugins/blob/master/plugins/check_users.c I can't possible see how -13 is possible. It seems the check uses 'who' to figure out the number of users. Who is part of coreutils-systemd: > # rpm -qf /usr/bin/who > coreutils-systemd-9.4-2.1.x86_64 Which is precisely what got added by https://bugzilla.opensuse.org/show_bug.cgi?id=1216148. Not saying it's related, but at this moment I don't have more clues.
The complete list of packages that got updated last night: > 2023-11-03 01:49:16|install|MicroOS-release-appliance|20231101-2572.1|x86_64||repo-oss|66d9b2f41d1b7b70c233b156ed71237340e2e8120aebae8997138ba7ce628c4c63f8ba106cb2a19053071ba9054f8829bc790fe280620af7c12893b16de2bf9c| > 2023-11-03 01:49:17|install|dracut|059+suse.511.g0bdb16ac-1.1|x86_64||repo-oss|91229ad8a68b5b9938d5d71c8247cad694bc8c49719fcb857be768f62f1836893ff7119b0f1dfb3a8ad83bbe91bcc6ef92a8d1fed9baf20b4716a9842f06896f| > 2023-11-03 01:49:18|install|grub2|2.12~rc1-8.1|x86_64||repo-oss|fe3cf5dc8ac6aba5eb1e4214fa1f27d88aca0b565acd987f53c35549f68b7121b7cbe30f25a9d59904ee6defe47a1356350cd995f8999a9b6e104344f98483be| > 2023-11-03 01:49:18|install|grub2-i386-pc|2.12~rc1-8.1|noarch||repo-oss|3e8dfad4e859521f58968edebe01edd539a56aee2fa5738e3c1ca602136ff005ebc0826131c148da0a9daaa565c1a08106fc15308c25f5bef54770d42007bf64| > 2023-11-03 01:49:19|install|libgpgme11|1.23.1-1.1|x86_64||repo-oss|14683838ad94829783aa600611b3c68661eabd4cd9283b6b8be3c58115f548fcac0d388df317a7fa6e8000c6c32efe33fd1319aa95e8ad97dd72a88cf9efff04| > 2023-11-03 01:49:19|install|monitoring-plugins-disk|2.3.3-1.1|x86_64||repo-oss|e2a42bf979d7d9d0b23019d02e7db66933e8cd3c79d1bfcedf454f0de3005afbdeb4893218de33706c70cf33d69abb84eb3491389d23646b4a8d3decf4c2b07c| > 2023-11-03 01:49:19|install|monitoring-plugins-load|2.3.3-1.1|x86_64||repo-oss|81b7e397d0ca2f9a2db84a943411f3a90bea6b29af5a159e247117aff0475689dd0288db7936b060b764657098edac2b17e7a4980ea788b05e17e307fffb8892| > 2023-11-03 01:49:19|install|monitoring-plugins-procs|2.3.3-1.1|x86_64||repo-oss|ddd31048aee6d42cad4effd4e5938f6fb8d91d9df125f69b9f0d84f50abc0834737fbf88ad61eaa18b812cde309a93f721094b521d95d1c3eae0f4ec3f420869| > 2023-11-03 01:49:19|install|monitoring-plugins-users|2.3.3-1.1|x86_64||repo-oss|8c063521869d4ad1ee1c2e409fd06ffa09cbe6f86ee3cf732a6213895796eba9c396a06ec53efac2b315d0920490e8f5bbd508da0396327432e800b5d0624ccb| > 2023-11-03 01:49:20|install|MicroOS-release|20231101-2572.1|x86_64||repo-oss|df6775f69060fbe0e999780c082cfdb96e490ec1aae3eb3d37692bb3e726092367d4c5d18f324f768b1258106bf148b91a911bba79fbf75acf013b1027df17fa| > 2023-11-03 01:49:21|install|grub2-x86_64-efi|2.12~rc1-8.1|noarch||repo-oss|7a76f65b88a3bc28b15dca721f2621e1db76ef205f8d2c24b074f7fff889c64896328e829fe8ad1497e7ba37a816a81fff612d75580af9f99883525a2707e880| > 2023-11-03 01:49:21|install|grub2-snapper-plugin|2.12~rc1-8.1|noarch||repo-oss|50614a390d61178fe01f67acb5ea57a6d857de1ef1a17f38fadfe16025745a792af71abd4569a1b31d95cbfabac81b1f2ec06b0fe69fe47b76fb4e3d993b3763| > 2023-11-03 01:49:21|install|grub2-i386-pc-extras|2.12~rc1-8.1|noarch||repo-oss|890361cbd5d93c23241e2a678eeac9ef854726a2a9c8e77912206054513170f99d90530b986cbc0e247798396f875272ca2bbbba5cf247e333d4c07fe433c273| > 2023-11-03 01:49:21|install|grub2-x86_64-efi-extras|2.12~rc1-8.1|noarch||repo-oss|bc7423130ddf1d337205797a8730694fb056fd90099bb1e24abdcd018ae91bca6ee4b6353007cf3d52a8013929003d55b1598fd0f90630295566361cd65b639d|
Reading the source it appears that this configures with utmpx, and utmp was just removed from TW.
(In reply to William Brown from comment #2) > Reading the source it appears that this configures with utmpx, and utmp was > just removed from TW. Good hint. At server:monitoring it still building for Tumbleweed with utmpx.h: https://build.opensuse.org/public/build/server:monitoring/openSUSE_Tumbleweed/x86_64/monitoring-plugins/_log > [ 26s] checking for utmpx.h... yes So I guess the build happens with utmpx support, while during runtime utmpx is not available. Debian 12: > # ls -l /var/run/utmp > -rw-rw-r-- 1 root utmp 1152 Nov 3 22:13 /var/run/utmp Tumbleweed: > # /var/run/utmp > -bash: /var/run/utmp: No such file or directory At the same time, I see https://build.opensuse.org/package/view_file/server:monitoring/monitoring-plugins/systemd-not-utmp.patch?expand=1 which should disable utmpx usage... maybe the patch is not complete now that utmpx is gone for good? I am just guessing, but maybe check_users is entering line 117 https://github.com/nagios-plugins/nagios-plugins/blob/master/plugins/check_users.c#L117 and that's what's causing -13 to be returned.
(In reply to William Brown from comment #2) > Reading the source it appears that this configures with utmpx, and utmp was > just removed from TW. utmp and utmpx are identical with glibc, and none of them can return an error. The biggest design mistake of this interface. (In reply to Julio González Gil from comment #3) > I am just guessing, but maybe check_users is entering line 117 > https://github.com/nagios-plugins/nagios-plugins/blob/master/plugins/ > check_users.c#L117 and that's what's causing -13 to be returned. The getutxent() functions don't return an error. Which of course does not mean that errno could be set from some internal function calls, but it's meaningless. -13 is EACCES. So maybe your process cannot read /run/systemd/sessions/ ? But this should be readable for everybody, else /usr/bin/who wouldn't work, too.
(In reply to Thorsten Kukuk from comment #4) > (In reply to William Brown from comment #2) > > Reading the source it appears that this configures with utmpx, and utmp was > > just removed from TW. > > utmp and utmpx are identical with glibc, and none of them can return an > error. The biggest design mistake of this interface. > > (In reply to Julio González Gil from comment #3) > > I am just guessing, but maybe check_users is entering line 117 > > https://github.com/nagios-plugins/nagios-plugins/blob/master/plugins/ > > check_users.c#L117 and that's what's causing -13 to be returned. > > The getutxent() functions don't return an error. Which of course does not > mean that errno could be set from some internal function calls, but it's > meaningless. > > -13 is EACCES. So maybe your process cannot read /run/systemd/sessions/ ? > But this should be readable for everybody, else /usr/bin/who wouldn't work, > too. Well, even if I call /usr/lib/nagios/plugins/check_users as root, I still get -13 as the number of users logged in, since the updated mentioned on c#1 The system is a standard openSUSE MicroOS, and the rights for /run/systemd/sessions: The directory itself: > microos:~ # ls -l /run/systemd/|grep sessions > drwxr-xr-x 2 root root 120 Nov 6 16:21 sessions The contents: > # ls -l /run/systemd/sessions/ > total 8 > -rw-r--r-- 1 root root 310 Nov 5 04:31 2 > prw------- 1 root root 0 Nov 5 04:31 2.ref > -rw-r--r-- 1 root root 320 Nov 6 16:21 3 > prw------- 1 root root 0 Nov 6 16:21 3.ref
Do you use AppArmor? It's the only tool I could imagine which blocks such access.
(In reply to Thorsten Kukuk from comment #6) > Do you use AppArmor? > It's the only tool I could imagine which blocks such access. Whatever openSUSE MicroOS is doing by default... so the answer seems to be "yes". And it seems the checks have a profile for it: > microos:~ # apparmor_status > apparmor module is loaded. > 64 profiles are loaded. > 64 profiles are in enforce mode. > /usr/bin/lessopen.sh > /usr/lib/nagios/plugins/check_disk > /usr/lib/nagios/plugins/check_load > /usr/lib/nagios/plugins/check_mem > /usr/lib/nagios/plugins/check_mem.pl > /usr/lib/nagios/plugins/check_procs > /usr/lib/nagios/plugins/check_users But why this would break suddenly because of the update? On the list of updates for that night, there are no apparmor changes, and IIUC the updates that got released for monitoring-plugins as part of the fix for https://bugzilla.opensuse.org/show_bug.cgi?id=1216148 should not cause this. In any case here are the contents of /etc/apparmor.d/usr.lib.nagios.plugins.check_users > #include <tunables/global> > /usr/lib/nagios/plugins/check_users { > #include <abstractions/base> > #include <abstractions/consoles> > #include <abstractions/wutmp> > /usr/lib/nagios/plugins/check_users rm, >} I am a complete ignorant when it comes to Apparmor but I guess this means access to utmp should be allowed? > microos:~ # cat /etc/apparmor.d/abstractions/wutmp > # ------------------------------------------------------------------ > # > # Copyright (C) 2002-2009 Novell/SUSE > # Copyright (C) 2009 Canonical Ltd. > # > # This program is free software; you can redistribute it and/or > # modify it under the terms of version 2 of the GNU General Public > # License published by the Free Software Foundation. > # > # ------------------------------------------------------------------ > > abi <abi/3.0>, > > # some services update wtmp, utmp, and lastlog with per-user > # connection information > /var/log/lastlog rwk, > /var/log/wtmp rwk, > /var/log/btmp rwk, > @{run}/utmp rwk, > > # Include additions to the abstraction > include if exists <abstractions/wutmp.d> But seems /run/utmp is gone, that matches what William Brown told at c#2. So I guess the problem is that utmp is still available for building (c#3), but not during runtime (so /run/utmp does not exist anymore during runtime and that causes EACCESS)?
(In reply to Julio González Gil from comment #7) > (In reply to Thorsten Kukuk from comment #6) > > Do you use AppArmor? > > It's the only tool I could imagine which blocks such access. > > Whatever openSUSE MicroOS is doing by default... so the answer seems to be > "yes". openSUSE MicroOS uses by default SELinux. > In any case here are the contents of > /etc/apparmor.d/usr.lib.nagios.plugins.check_users > > > #include <tunables/global> > > /usr/lib/nagios/plugins/check_users { > > #include <abstractions/base> > > #include <abstractions/consoles> > > #include <abstractions/wutmp> > > /usr/lib/nagios/plugins/check_users rm, > >} > > I am a complete ignorant when it comes to Apparmor but I guess this means > access to utmp should be allowed? As announced several times: openSUSE Tumbleweed, MicroOS and variants don't use utmp anymore. Already for weeks. > But seems /run/utmp is gone, that matches what William Brown told at c#2. Of course is /run/utmp gone, we announce that since 9 month or so... > So I guess the problem is that utmp is still available for building (c#3), > but not during runtime (so /run/utmp does not exist anymore during runtime > and that causes EACCESS)? No, that's not the problem. Somebody needs to enhance the apparmor utmp file list and allow reading /run/systemd/sessions/.
(In reply to Thorsten Kukuk from comment #8) > (In reply to Julio González Gil from comment #7) > > (In reply to Thorsten Kukuk from comment #6) > > > Do you use AppArmor? > > > It's the only tool I could imagine which blocks such access. > > > > Whatever openSUSE MicroOS is doing by default... so the answer seems to be > > "yes". > > openSUSE MicroOS uses by default SELinux. That's odd. This is a very old instance that I created maybe 2 years ago or more (basically to replace CoreOS after it was cancelled), but other than installing monitoring and the Java Runtime Environment to run a Jenkins agent, I never played around with Apparmor or SELinux. It basically got updated automatically all this time, with one exception where I had to rollbacik because of a bug on the kernel (IIRC), and the two bugs I reported for monitoring (including this one). Maybe openSUSE MicroOS got changes from Apparmor to SELinux, but old instances didn't get migrated to SELinux. No idea. > > In any case here are the contents of > > /etc/apparmor.d/usr.lib.nagios.plugins.check_users > > > > > #include <tunables/global> > > > /usr/lib/nagios/plugins/check_users { > > > #include <abstractions/base> > > > #include <abstractions/consoles> > > > #include <abstractions/wutmp> > > > /usr/lib/nagios/plugins/check_users rm, > > >} > > > > I am a complete ignorant when it comes to Apparmor but I guess this means > > access to utmp should be allowed? > > As announced several times: openSUSE Tumbleweed, MicroOS and variants don't > use utmp anymore. Already for weeks. Ok, it's a bit confusing that the build log mentions, to this day: > [ 26s] checking for utmpx.h... yes Hopefull that only means it detected the headers file, but not that it will build with utmp support. > > So I guess the problem is that utmp is still available for building (c#3), > > but not during runtime (so /run/utmp does not exist anymore during runtime > > and that causes EACCESS)? > > No, that's not the problem. Somebody needs to enhance the apparmor utmp file > list and allow reading /run/systemd/sessions/. I never touched apparmour, but I will try to have a quick look as part of the Hack Week (hopefully it's just a matter of fixing that file, and then preparing SRs). If I can't quickly fix it, I guess I will just redeploy this openSUSE MicroOS and I guess it will start using SELinux. Will report the outcome here later.
Upstream issue: https://gitlab.com/apparmor/apparmor/-/issues/360 If they are OK with the solution, I will submit a Merge Request and then prepare a patch for Factory until the new version arrives.
For now, my own workaround was creating a file at /etc/apparmor.d/abstractions/sessions and then restarting apparmor. Content of the file: > /run/systemd/sessions/ r, That fixes the issue until we get a reply from upstream. If it takes too much, then I guess I will prepare SRs to patch profiles/apparmor.d/abstractions/wutmp until the question is solved. Are you OK with the plan, Thorsten?
Removing the needinfo, as upstream already replied. I will prepare the fix for upstream soon, and then a patch for our packages until we get the new release from upstream.
Upstream MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1121 OBS SR: https://build.opensuse.org/request/show/1124273
This is an autogenerated message for OBS integration: This bug (1216878) was mentioned in https://build.opensuse.org/request/show/1124276 Factory / apparmor