Bug 1192996 - Changed 'lsblk' behavior breaks OpenQA tests and customer scripts
Summary: Changed 'lsblk' behavior breaks OpenQA tests and customer scripts
Status: NEW
: 1192971 1192972 (view as bug list)
Alias: None
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Basesystem (show other bugs)
Version: Current
Hardware: All openSUSE Tumbleweed
: P5 - None : Major (vote)
Target Milestone: ---
Assignee: Stanislav Brabec
QA Contact: E-mail List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-11-23 13:43 UTC by Stefan Hundhammer
Modified: 2024-02-25 14:32 UTC (History)
8 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Hundhammer 2021-11-23 13:43:53 UTC
Command: lsblk
Package: util-linux-systemd-2.37

Affected Code Streams: Tumbleweed, SLE-15-SP4, Leap 15.4

This bug was cloned / split off from bug #1192972.


PROBLEM
=======

The 'lsblk' command changed its behavior: It used to report only the first / primary mount point of a filesystem; now it reports ALL mount points.

This is a problem because 'lsblk' used to be a simple check if a directory is on a separate filesystem:

    lsblk | grep '/home'

But now this will also show a Btrfs subvolume with that path. This breaks a lot of OpenQA tests, and it will also break a lot of customer / end user scripts.



WORKAROUNDS
===========

The man page says that by default it does the same as if the '--fs' option was specified which alledgedly does the same as

    lsblk -o NAME,FSTYPE,FSVER,LABEL,UUID,FSAVAIL,FSUSE%,MOUNTPOINTS

so a script using this could also use the singular form MOUNTPOINT to list only one, but that does not list the primary one, but the one that was last mounted (?):


[sh @ balrog-tw-dev] ~ 22 % lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
sda      8:0    0   30G  0 disk 
├─sda1   8:1    0    8M  0 part 
├─sda2   8:2    0   28G  0 part /mnt
│                               /
└─sda3   8:3    0    2G  0 part [SWAP]
sdb      8:16   0  300M  0 disk 
sr0     11:0    1 1024M  0 rom  


[sh @ balrog-tw-dev] ~ 22 % lsblk -o NAME,MAJ:MIN,RM,SIZE,RO,TYPE,MOUNTPOINT 
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda      8:0    0   30G  0 disk 
├─sda1   8:1    0    8M  0 part 
├─sda2   8:2    0   28G  0 part /mnt
└─sda3   8:3    0    2G  0 part [SWAP]
sdb      8:16   0  300M  0 disk 
sr0     11:0    1 1024M  0 rom  


Notice that it shows /mnt, an additional mount of /dev/sda2, not / as I would expect. So this is not really helpful.
Comment 1 Stefan Hundhammer 2021-11-23 13:44:51 UTC
See also bug #1192972 and bug #1192971
Comment 2 Stefan Hundhammer 2021-11-23 13:49:05 UTC
I assume this change comes from upstream, so reverting it may not be an option for us.

I split off this bug from the other ones because I expect quite a number of bugs as a consequence. It is up to the QA team / the release managers and the maintainers of 'lsblk' to make a decision about this; it may end up as WONTFIX or similar. The QA team and customers relying on the old behavior may have to find a suitable workaround.

But we, the YaST team, need a collecting point for all incoming bugs for this problem, and it needs to be clear what the root cause is.
Comment 3 Stefan Hundhammer 2021-11-23 13:50:57 UTC
For some possible workarounds and related tools, see

https://github.com/shundhammer/qdirstat/blob/master/doc/Shadowed-by-Mount.md#2a-caveat-btrfs
Comment 4 Stefan Hundhammer 2021-11-23 13:54:26 UTC
[sh @ balrog-tw-dev] ~ 20 % osc maintainer -e util-linux-systemd
Defined in package: Base:System/util-linux 
  bugowner of util-linux-systemd : 
   sbrabec@suse.com

  maintainer of util-linux-systemd : 
   meissner@suse.com, sbrabec@suse.com, jengelh@inai.de
Comment 5 Stefan Hundhammer 2021-11-23 14:59:14 UTC
*** Bug 1192972 has been marked as a duplicate of this bug. ***
Comment 6 Stefan Hundhammer 2021-11-23 15:00:23 UTC
*** Bug 1192971 has been marked as a duplicate of this bug. ***
Comment 7 Dominique Leuenberger 2021-11-23 15:31:48 UTC
Related to
https://progress.opensuse.org/issues/102795
Comment 8 Imobach Gonzalez Sosa 2021-11-23 17:21:17 UTC
For the records, I have checked that this change should not affect yast2-users, which uses lsblk too[1].

[1] https://github.com/yast/yast-users/blob/0e361ce62ebaf4bac1e0bb6f44a19f9c8c93c8b2/src/lib/users/leaf_blk_device.rb#L63-L64
Comment 9 Stefan Hundhammer 2021-11-24 09:23:56 UTC
The upstream release notes mention this as a new feature:

https://mirrors.edge.kernel.org/pub/linux/utils/util-linux/v2.37/v2.37-ReleaseNotes
Comment 10 Stefan Hundhammer 2021-11-24 09:32:42 UTC
LSBLK(8)              System Administration             LSBLK(8)

NAME
       lsblk - list block devices

SYNOPSIS
       lsblk [options] [device...]

DESCRIPTION

...
...
       ... This is the reason why lsblk provides MOUNTPOINT
       and MOUNTPOINTS (pl.) columns. The column MOUNTPOINT
       displays only one mount point (usually the last mounted
       instance of the filesystem)


-----------------------------------------

This behavior is indeed documented, but still I consider this broken by design: If it tells me only ONE mount point, I want to know the primary one, i.e. the first one that was mounted, not the most recent one; that is completely useless information in all circumstances that I can think of.

I know that in the kernel world, they think that all mount points are created equal, and that there is no such thing as a primary one; but in the real world, there certainly is, and that is always the one that was first mounted.

Even more so in the context of Btrfs: It clearly has a main mount point, and then there are the subvolumes with their mount points. At least in that case I'd expect  that one to be chosen for MOUNTPOINT.
Comment 11 Stanislav Brabec 2024-02-17 22:45:33 UTC
Actually, the kernel interface does not provide sufficient information to detect "first mounted" volume.

It would be too complicated to implement such logic. One would need to create mount log somewhere (that is capable to write even during early boot).

The only easy thing I can imagine: write a shortest MOUNTPOINT path that was found. What do you think? But this will easily fail. Mount /home, then the same as /mnt, it will report /mnt.
Comment 12 Stefan Hundhammer 2024-02-19 08:51:25 UTC
Using the shortest path is a very pragmatic approach that probably works well in most cases.

It could work even better in combination with a list of well-known system directories that could get priority if there is any doubt; a list like

["/", "/usr", "/var", "/home",
 "/dev", "/proc", "/sys", "/run"]


I wouldn't include ["/bin", "/sbin", "/lib", "/lib64"] because they are becoming obsolete with the usr-merge, and in a traditional system without the usr-merge mounting anything there would be asking for trouble.
Comment 13 Jan Engelhardt 2024-02-19 13:00:56 UTC
>'lsblk' used to be a simple check if a directory is on a separate filesystem

You cannot determine if /dev/shm is on a separate filesystem with lsblk.
You must use something like `stat -c %d` in any case.

>it will also break a lot of customer / end user scripts

As long as lsblk outputs Unicode tree graphics


>Actually, the kernel interface does not provide sufficient information to detect "first mounted" volume. It would be too complicated to implement such logic.

If the MNTID field (1st column of /proc/self/mountinfo), or a hypothetical new 12th field, were to be a strictly monotonically increasing counter (atomic_ulong), that would be rather easy to tell. You could just make it a boot-relative ktime. 8 bytes extra memory per entry, easy.
Comment 14 Stanislav Brabec 2024-02-25 14:32:00 UTC
Using the shortest path is a very pragmatic approach, but it breaks in some typical double mounts done by advanced users (and double mounts are used mostly by advanced users):

/home vs. /mnt. One wants to see /home, will see /mnt

/home/snapshots/2024-01-01 vs. /mnt/1. One wants to see /home/snapshots/2024-01-01, but will see /mnt/1

Even worse, the kernel mount interface is not ideal for btrfs, and it could be confused by bind mounts combined with subvolume mounts.

Adding yet another heuristic to this heuristic could produce unexpected results.

Hopefully, a new API exists: mountfd. But it is still buggy and disabled in all recent products including Tumbleweed.