Bug 1211725

The problem persists for openSUSE Leap 15.5 after a fresh install from openSUSE-Leap-15.5-DVD-x86_64-Build491.1-Media.iso and after updating via

zypper patch -l --no-recommends --with-update

Thanks for your report. This is also reproducible in Tumbleweed, so I've created an issue upstream to see if it's a bug or an (un)known feature.

https://github.com/systemd/systemd/issues/28657

Upstream answered that this is the expected behavior, but it was not documented, so the systemd-fsck@.service man page was updated (https://github.com/systemd/systemd/commit/000680a68d).

Thanks for looking into this. So what should people do instead? Is it a good idea to add "/sbin/fsck LABEL=mirror" to /etc/init.d/boot.local? (Note: I have a shell script in /etc/cron.daily mirroring / to the partition labeled "mirror".)

PS: If at all, the line in /etc/init.d/boot.local should read

/sbin/fsck -p LABEL=mirror

You can create a service requiring the fsck on the desired partition. Something like:

```

[Unit]
Description=fsck /dev/sda1
Requires=systemd-fsck@dev-sda1.service
# using your label shuld be something like:
#Requires=systemd-fsck@dev-disk-by\x2dlabel-mirror.service

[Service]
ExecStart=/bin/true

[Install]
WantedBy=multi-user.target

```

If you need to run this daily, you can create a timer.

(In reply to Stephan Wefing from comment #0)
> According to "man systemd-fsck@.service", setting the 6th field of the
> corresponding line in /etc/fstab to a value greater than 0 should suffice to
> instantiate systemd-fsck@.service at boot time.

Not exactly, the man page doesn't state that the service is instantiated at boot time. In fact systemd-fsck@xxx.service is instantiated when the mount unit requiring it is started. That's the reason why you can't find it when "noauto" is specified.

(In reply to Stephan Wefing from comment #4)
> Thanks for looking into this. So what should people do instead? 

Normally nothing, instances of systemd-fsck@.service are automatically started when the mount units are.

> Is it a good idea to add "/sbin/fsck LABEL=mirror" to /etc/init.d/boot.local? 

As written previous you shouldn't have to do anything, filesystems are checked just before they're mounted.

And using /etc/init.d/boot.local is not a good idea as this init script (as any other ones) are deprecated.

(In reply to Franck Bui from comment #7)
> (In reply to Stephan Wefing from comment #0)
> > According to "man systemd-fsck@.service", setting the 6th field of the
> > corresponding line in /etc/fstab to a value greater than 0 should suffice to
> > instantiate systemd-fsck@.service at boot time.
> 
> Not exactly, the man page doesn't state that the service is instantiated at
> boot time. In fact systemd-fsck@xxx.service is instantiated when the mount
> unit requiring it is started. That's the reason why you can't find it when
> "noauto" is specified.

Hmm, in the man page of systemd-fsck@.service, under "DESCRIPTION", I read:

"These services are started at boot if passno in /etc/fstab for the file system is set to a value greater than zero."

In the man page of fstab,  under "DESCRIPTION", I read:

"The sixth field (fs_passno).
       This field is used by fsck(8) to determine the order in which
       filesystem checks are done at boot time."

Since fs_passno is 2 in my case, I conclude, that a filesystem check should be carried out if it is due. (man pages quoted refer to an up-to-date leap 15.4 system.)

(In reply to Stephan Wefing from comment #9)
> (In reply to Franck Bui from comment #7)
> > (In reply to Stephan Wefing from comment #0)
> > > According to "man systemd-fsck@.service", setting the 6th field of the
> > > corresponding line in /etc/fstab to a value greater than 0 should suffice to
> > > instantiate systemd-fsck@.service at boot time.
> > 
> > Not exactly, the man page doesn't state that the service is instantiated at
> > boot time. In fact systemd-fsck@xxx.service is instantiated when the mount
> > unit requiring it is started. That's the reason why you can't find it when
> > "noauto" is specified.
> 
> Hmm, in the man page of systemd-fsck@.service, under "DESCRIPTION", I read:
> 
> "These services are started at boot if passno in /etc/fstab for the file
> system is set to a value greater than zero."
> 
> In the man page of fstab,  under "DESCRIPTION", I read:
> 
> "The sixth field (fs_passno).
>        This field is used by fsck(8) to determine the order in which
>        filesystem checks are done at boot time."
> 
> Since fs_passno is 2 in my case, I conclude, that a filesystem check should
> be carried out if it is due. (man pages quoted refer to an up-to-date leap
> 15.4 system.)

Since fs_passno is 2 in my case, I conclude, that a filesystem check should
be carried out at boot time if it is due.

(In reply to Franck Bui from comment #8)
 
> Normally nothing, instances of systemd-fsck@.service are automatically
> started when the mount units are.

Upon mounting /mirror manually, only a message was written to the journal that a filesystem check is due, but the check was not carried out. This resulted in /mirror not being checked for years.

(In reply to Stephan Wefing from comment #9)
> Hmm, in the man page of systemd-fsck@.service, under "DESCRIPTION", I read:
> 
> "These services are started at boot if passno in /etc/fstab for the file
> system is set to a value greater than zero."

Ok, that's the part which is fixed by Antonio's patch.

> In the man page of fstab,  under "DESCRIPTION", I read:
> 
> "The sixth field (fs_passno).
>        This field is used by fsck(8) to determine the order in which
>        filesystem checks are done at boot time."
> 
> Since fs_passno is 2 in my case, I conclude, that a filesystem check should
> be carried out if it is due. (man pages quoted refer to an up-to-date leap
> 15.4 system.)

Only if the associated filesystem is mounted at boot time, which is not the case with "noauto".

(In reply to Stephan Wefing from comment #11)
> Upon mounting /mirror manually, only a message was written to the journal

Well if you mount it manually you loose any additional steps that would be done by systemd automatically.

You need to run `systemctl start xxx.mount` instead.

I think this can be closed since the documentation has been clarified (see comment #3).

Feel free to re-open if you feel that's not the case.

(In reply to Franck Bui from comment #12)

> Only if the associated filesystem is mounted at boot time, which is not the
> case with "noauto".

This pre-condition / this dependency on "noauto" is not mentioned in the man page
of fstab:

"
The sixth field (fs_passno).
       This field is used by fsck(8) to determine the order in which filesystem
       checks are done at boot time. The root filesystem should be specified 
       with a fs_passno of 1. Other filesystems should have a fs_passno of 2.
       Filesystems within a drive will be checked sequentially, but filesystems
       on different drives will be checked at the same time to utilize 
       parallelism available in the hardware. Defaults to zero (don’t check the
       filesystem) if not present.
"

(man page quoted refers to an up-to-date leap 15.4 system.)

From fstab(5) man page:

       noauto
           do not mount when mount -a is given (e.g., at boot time)

Did you run `mount -a` when you mounted /mirror manually ?

(In reply to Franck Bui from comment #16)
> From fstab(5) man page:
> 
>        noauto
>            do not mount when mount -a is given (e.g., at boot time)
> 
> Did you run `mount -a` when you mounted /mirror manually ?

No, I use 'mount /mirror'.

So it's documented in the fstab man page that fs_passno=2 is effective at boot time (ie not later at runtime). The fact that you additionally used "noauto" option makes the filesystem skipped at boot time.

You expect that running "mount /mirror" at runtime should honor "fs_passno=2" which cannot be expected according to the doc (see above).

I suggested you to run "systemd start mirror.mount" instead because systemd, when parsing /etc/fstab, stored the fact that the filesystem should be fscked in the mount unit.

So I still think that this bug is invalid.

If you still think that the behavior of "mount /mirror" is wrong, then I would suggest to talk to upstream directly since SUSE doesn't do anything differently in this regard. Also systemd is not involved here so I would suggest you to contact util-linux instead.

Hope that makes sense.

(In reply to Franck Bui from comment #18)
> You expect that running "mount /mirror" at runtime should honor
> "fs_passno=2" which cannot be expected according to the doc (see above).

No, this is a misunderstanding: I'm talking about missing filesystem checks at boot time, this was my motivation to file this report. Like the "fstab" man page, the "mount" man page does not establish a link between mounting a filesystem and checking a filesystem. Hence, I do not expect a filesystem to be checked if it is mounted via the "mount" command.

> So it's documented in the fstab man page that fs_passno=2 is effective at
> boot time (ie not later at runtime). The fact that you additionally used
> "noauto" option makes the filesystem skipped at boot time.

According to the fstab man page (fourth field), "skipped" can only refer to the mount operation at boot time, not to filesystem checking at boot time. filesystem checks are only mentioned in the description of the sixth field, but not in the description of the fourth field - and vice versa: the mount operation is only mentioned in the description of the fourth field, but not in the description of the sixth field. 

Hence, the "fstab" man page does not link the check of a filesystem at boot time to being mounted at boot time. Hence, I expect a filesystem to be checked at boot time even if "noauto" is specified in the fourth field. This expectation was met until some point in the past. Before that point, my /mirror filesystem was checked regularly at boot time even though "noauto" was set in the fourth field. Unfortunately, this no longer holds.

Hence, my impression is that the logic was changed but the fstab man page has not been updated accordingly. Hence, not only the "systemd-fsck@.service" man page has to be updated (which has already been done, see comment 3), but also the "fstab" man page.

(In reply to Stephan Wefing from comment #19)
> Hence, my impression is that the logic was changed but the fstab man page
> has not been updated accordingly. Hence, not only the
> "systemd-fsck@.service" man page has to be updated (which has already been
> done, see comment 3), but also the "fstab" man page.

ok then let's reassign this bug to our util-linux maintainer (although you might contact upstream directly since your concern is not SUSE specific) .

Stanislav, could you take over ? the fstab man page needs to reflect the fact that fsck is no more run on filesystems with "noauto" option. This is true since the introduction of systemd.

Perhaps something to deal with upstream directly.

SUSE-RU-2024:0181-1: An update that has four fixes can now be installed.

Category: recommended (moderate)
Bug References: 1211576, 1211725, 1212207, 1215241
Sources used:
SUSE Linux Enterprise Software Development Kit 12 SP5 (src): systemd-228-157.57.1
SUSE Linux Enterprise High Performance Computing 12 SP5 (src): systemd-228-157.57.1
SUSE Linux Enterprise Server 12 SP5 (src): systemd-228-157.57.1
SUSE Linux Enterprise Server for SAP Applications 12 SP5 (src): systemd-228-157.57.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.

The new version of util-linux has the man page fixed.

Too minor to justify an update for released products. But maybe we can wait for the next update of util-linux.

My mistake in the comment 27: The systemd man page is updated, not util-linux.

Digging deeper, This problem does not affect util-linux fsck at all.

systemd-fsck@.service does not call "fsck -A", but it has its own implementation /usr/lib/systemd/systemd-fsck that exhibits reported behavior.

utul-linux "fsck -A" uses only fs_passno and ignores auto/noauto in the options field. (But it skips non-existing devices with auto as fs.)

So there is no need to adjust util-linux.

Yes, it's a bit confusing, but there are two independent implementations of fsck with a slightly different behavior:

util-linux/disk-utils/fsck.c
systemd/src/fsck/fsck.c

As fstab.5 man page is part of util-linux, it reflects behavior of util-linux fsck, not the systemd-fsck.


So I either propose to keep the man page as it is, or add mention that there exists systemd-fsck that may interpret fstab differently.

After a discussion with Karel Žák and Lennart Poettering, upstream util-linux got a notice.


https://github.com/util-linux/util-linux/commit/9cb7b7671d903573d6c3b9d8112ec13953cdcdc6

The discussion: https://marc.info/?l=util-linux-ng&m=170799010632727&w=2

Closing the bug as INVALID. It does not affect fsck. It is a standard (and documented) behavior of systemd-fsck.

To prevent mis-expectations, future versions of util-linux will have a notice about systemd-fsck and its behavior in fsck(8) and fstab(5).