Bugzilla – Bug 1213595
Live ISOs fail to boot due to wrong by-label links
Last modified: 2024-07-16 12:31:14 UTC
Created attachment 868398 [details] 7z File for some logs Hi, I downloaded OpenSUSE Leap ISO (openSUSE-Leap-15.5-KDE-Live-x86_64-Build10.40-Media.iso) at 08/06/2023 (as soon as released) then I “dd” to USB stick and plugged to computer and it worked fine. At first boot, it did create the persistent storage cow (boot.log : [ 22.002982] dracut-initqueue[877]: mke2fs 1.46.4 (18-Aug-2021) ) Few days later I download again this time build number was different (openSUSE-Leap-15.5-KDE-Live-x86_64-Build10.57-Media.iso). At first boot, it did not create persistent storage. dracut says (boot.log : [ 21.489260] dracut-initqueue[769]: Warning: Falling back to temporary overlay) same as the new release (openSUSE-Leap-15.5-KDE-Live-x86_64-Build10.73-Media.iso) Any release after 10.40 does not creates persistent storage (neither 10.53 nor 10.73) I tried with different CPU (AMD, Core Duo, I5) and different USB stick (4GB 8GB 16GB and 32GB) even with different port (USB2 USB3) with different USB s and so on. No matter what I tried don't work. Releases after 10.40 (10.57 or 10.73) do not creates the persistent storage. So I open discussion on the forums.opensuse.org and someone suggested that I need to open bug report. and he/she wrote "It is not something you can fix without recreating initrd. The script that is used here comes from kiwi dracut module.". I also add the 7zip file for each ISO "....10.40", "...10.57", "...10.73" that contains some log files. "boot.log" : "sudo cp /var/log/boot.log" "boot.msg" : "cp /var/log/boot.msg" "dmesg.txt" : Output of "dmesg" right after the first boot "fdisk.txt" : 0utput of "fdisk -l" right after first boot "journal.txt" : Output of "sudo journalctl --no-pager --full -b" Thank you very much.
Confirmed, this is also visible in openQA: https://openqa.opensuse.org/tests/3456098 [ 9.243802] dracut-initqueue[882]: mount: /run/overlay/overlayfs: can't find LABEL="cow". [ 9.254383] dracut-initqueue[887]: This disk is currently in use - repartitioning is probably a bad idea. [ 9.254420] dracut-initqueue[887]: It's recommended to umount all file systems, and swapoff all swap [ 9.254442] dracut-initqueue[887]: partitions on this disk. [ 9.254459] dracut-initqueue[887]: The device contains 'iso9660' signature and it may remain on the device. It is recommended to wipe the device with wipefs(8) or fdisk --wipe, in order to avoid possible collisions. [ 9.291225] dracut-initqueue[900]: mke2fs 1.46.4 (18-Aug-2021) [ 9.291387] dracut-initqueue[900]: /dev/sda3 is apparently in use by the system; will not make a filesystem here! [ 9.291630] dracut-initqueue[813]: Warning: Failed to setup persistent write space ! [ 9.291667] dracut-initqueue[813]: Warning: Falling back to temporary overlay The test passes though because it doesn't (yet) verify persistence, only that everything works. Probably a bug in kiwi, I'll collect some info to confirm that.
According to blkid, both /dev/sda and /dev/sda1 have LABEL="openSUSE_Leap_15.5_KDE_Live" and udev links sda into /dev/disk/by-label. This means that the whole disk is mounted, not just the first partition. Can it be avoided that the label is also visible on /dev/sda? Any idea what could cause this behaviour change? At least in the current TW snapshot it appears to be working.
Since release of Build12.x the bug disappered it it working now (Build13.8)
openSUSE-Leap-15.5-KDE-Live-x86_64-Build13.8-Media.iso works fine
Looks like this is broken again somehow, all openQA runs with 15.5 and 15.6 isos booted off USB fail: https://openqa.opensuse.org/tests/3992142#step/finish_desktop/2 I booted the system with "debug console=ttyS0 systemd.log_target=console rd.udev.log_level=debug", fortunately it still fails. Log is attached. It is visible that udev prefers the by-label link to sda instead of sda1 for some reason, even though https://bugzilla.suse.com/show_bug.cgi?id=1213185 implies that the fix for https://github.com/systemd/systemd/issues/28468 got backported. dev-disk-by\x2duuid-2024\x2d02\x2d21\x2d09\x2d57\x2d45\x2d00.device: Device dev-disk-by\x2duuid-2024\x2d02\x2d21\x2d09\x2d57\x2d45\x2d00.device appeared twice with different sysfs paths /sys/devices/pci0000:00/0000:00:09.0/usb4/4-1/4-1:1.0/host3/target3:0:0/3:0:0:0/block/sda and /sys/devices/pci0000:00/0000:00:09.0/usb4/4-1/4-1:1.0/host3/target3:0:0/3:0:0:0/block/sda/sda1, ignoring the latter. dev-disk-by\x2dlabel-openSUSE_Leap_15.5_Rescue_CD.device: Device dev-disk-by\x2dlabel-openSUSE_Leap_15.5_Rescue_CD.device appeared twice with different sysfs paths /sys/devices/pci0000:00/0000:00:09.0/usb4/4-1/4-1:1.0/host3/target3:0:0/3:0:0:0/block/sda and /sys/devices/pci0000:00/0000:00:09.0/usb4/4-1/4-1:1.0/host3/target3:0:0/3:0:0:0/block/sda/sda1, ignoring the latter.
This is an autogenerated message for OBS integration: This bug (1213595) was mentioned in https://build.opensuse.org/request/show/1162956 Factory / python-kiwi
After some investigation (and offline discussions) it appears that the problem lives in kiwi. It operates on the disk containing the live OS (to create a third partition) and this appears to generates some REMOVE uevents for the existing partitions. This has the side effect to temporarily modify the by-label symlink. Reassigning to Marcus.
Yep and the fix for this was merged upstream and package updates for Factory/TW ALP and SLES/Leap has been created. Thanks for your help in debugging the issue
Many thanks!
Should be fixed now
SUSE-RU-2024:1371-1: An update that has four fixes can now be installed. Category: recommended (moderate) Bug References: 1207128, 1213595, 1219074, 1221915 Maintenance Incident: [SUSE:Maintenance:33219](https://smelt.suse.de/incident/33219/) Sources used: openSUSE Leap 15.5 (src): python-kiwi-9.24.43-150100.3.76.1 SUSE Linux Enterprise Micro for Rancher 5.3 (src): python-kiwi-9.24.43-150100.3.76.1 SUSE Linux Enterprise Micro 5.3 (src): python-kiwi-9.24.43-150100.3.76.1 SUSE Linux Enterprise Micro for Rancher 5.4 (src): python-kiwi-9.24.43-150100.3.76.1 SUSE Linux Enterprise Micro 5.4 (src): python-kiwi-9.24.43-150100.3.76.1 SUSE Linux Enterprise Micro 5.5 (src): python-kiwi-9.24.43-150100.3.76.1 Development Tools Module 15-SP5 (src): python-kiwi-9.24.43-150100.3.76.1 SUSE Linux Enterprise Micro 5.1 (src): python-kiwi-9.24.43-150100.3.76.1 SUSE Linux Enterprise Micro 5.2 (src): python-kiwi-9.24.43-150100.3.76.1 SUSE Linux Enterprise Micro for Rancher 5.2 (src): python-kiwi-9.24.43-150100.3.76.1 openSUSE Leap Micro 5.3 (src): python-kiwi-9.24.43-150100.3.76.1 openSUSE Leap Micro 5.4 (src): python-kiwi-9.24.43-150100.3.76.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
SUSE-RU-2024:1371-2: An update that has four fixes can now be installed. Category: recommended (moderate) Bug References: 1207128, 1213595, 1219074, 1221915 Maintenance Incident: [SUSE:Maintenance:33219](https://smelt.suse.de/incident/33219/) Sources used: SUSE Linux Enterprise Micro 5.5 (src): python-kiwi-9.24.43-150100.3.76.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.