Bug 1216673

Summary: installer doesn't allow for non-standard boot configurations
Product: [openSUSE] openSUSE Tumbleweed Reporter: Bret Towe <bret.towe>
Component: BootloaderAssignee: Matthias Brugger <mbrugger>
Status: NEW --- QA Contact: E-mail List <qa-bugs>
Severity: Enhancement    
Priority: P5 - None CC: ancor, bret.towe, guillaume.gardet, guillaume.gardet, ivan.ivanov, jreidinger, mbrugger
Version: CurrentFlags: msuchanek: needinfo? (mbrugger)
msuchanek: needinfo? (guillaume.gardet)
Target Milestone: ---   
Hardware: Other   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: Step to add a partition

Description Bret Towe 2023-10-27 21:38:58 UTC
while trying to get opensuse working on the pine64 soquartz compute modules on the soblade I found some issues with the installer.
the soquartz/soblade has a few requirements to boot currently

the first item is uboot has to be on an emmc or sd card it can't be on the nvme drive the soblade supports.

second is the kernel and initramfs files needs to be on a drive that uboot sees at boot.
grub doesn't see any further drives when it loads either as it relies on uboot to see the hardware.

so the first item the installer needs to worry about is partitions need to allow setting an offset at the beginning
so you can avoid overwriting the uboot already installed on the device.
side note an option to allow changing partition size already in the layout instead of having to delete and readd would be nice...
I worked around the overwrite by not touching the 1st partition and using it as /boot/efi
as the offset needed was already set on the card via other means.

the second problem is if you move the /boot partition away from the root os drive it no longer lets you have snapshots so being able to use microos or transactional server options go out the window.
for example working drive layout looks like the below:
emmc1 (emmc chip)
32mb 'free space' holding uboot (min 16mb required) 
partition 1 - /boot/efi (512mb) 
partition 2 - /boot rest of drive (7+gb)

nvme0
partition 1 - lvm with root os etc
Comment 1 Stefan Hundhammer 2023-10-28 14:42:24 UTC
"Installer doesn't allow for non-standard boot configurations" - that is not a problem, it's a statement of fact. And it's not even true, either.

There are certain scenarios (lots of them!) that are supported on many architectures, and others that are not. There is no way that the YaST installer can support every conceivable scenario, even the most exotic ones. At some point, manual interaction becomes required.

We also cannot limit all other hardware / other scenarios to the very specific requirements of one very special hardware.


AFAICS, this is really a feature request for direct out-of-the-box support of such a very special hardware with very specific requirements.

So, what specific hardware is it? What architecture? What CPU, what chipset / mainboard? If our hardware enablement team team has access to such hardware, there may be a chance to support it. If not, then probably not.
Comment 2 Bret Towe 2023-10-28 16:09:08 UTC
> AFAICS, this is really a feature request for direct out-of-the-box support of such a very special hardware with very specific requirements.

it's a feature request to allow better support for any device that can't have root os on the boot device for space (or performance) reasons.
as the installer does not allow for snapshots on /boot if its not on the same device as the root os so I believe this problem could be hit on x86 systems also its just its a rare thing to do in that space
I could see someone wanting to say do local to device /boot but the root os is on an networked iSCSI device that only is accessible after the kernel and network is up

> So, what specific hardware is it? What architecture? What CPU, what chipset / mainboard? If our hardware enablement team team has access to such hardware, there may be a chance to support it. If not, then probably not.

I stated the hardware in the first line of my comment
here is link to the hardware to purchase:
https://pine64.com/product-category/soquartz/
link to wiki specs/info:
https://wiki.pine64.org/wiki/SOQuartz

tldr:
arm based system using a Rockchip RK3566 SoC
Comment 3 Bret Towe 2023-10-28 16:20:01 UTC
and i should also note that I did open bug #1216674 requesting a u-boot package to support this kit
as the u-boot package is really only item 'missing' from opensuse to support this hardware
Comment 4 Stefan Hundhammer 2023-10-30 09:43:37 UTC
I just asked our YaST bootloader expert Josef Reidinger, and he told me that support for U-Boot comes indirectly via Grub2 (chain booting if I understood him correctly). So that part is nothing that should need any change in YaST.

As for partitioning requirements: In many (not all) cases, it's the exact opposite, i.e. the root filesystem is required to be on the boot disk.

This is one of those cases where manual interaction might be required; the YaST storage proposal is just that, a proposal. If that doesn't fit your needs, you can use the "guided setup" for some more human control, or the expert partitioner for total control.
Comment 5 Stefan Hundhammer 2023-10-30 09:45:25 UTC
Reassigning to one of our ARM64 experts, Michal Suchanek.

Maybe there is even a solution for this specific hardware.
Comment 6 Ancor Gonzalez Sosa 2023-10-30 10:43:37 UTC
(In reply to Bret Towe from comment #2)
> 
> it's a feature request to allow better support for any device that can't
> have root os on the boot device for space (or performance) reasons.
> as the installer does not allow for snapshots on /boot if its not on the
> same device as the root os so I believe this problem could be hit on x86
> systems also its just its a rare thing to do in that space

As far as I understand, the installer enforces /boot to be part of root file-system when snapshots are used because that's a requisite for the standard snapshots tools we ship in the distribution.

I guess the configuration of Snapper, transactional-update, the bootloader and whatnot can be tweaked to work in that environment with separate /boot and / file-system. But I think the default configuration of all those tools assume everything is hosted in the same file-system. That's why the installer enforces that.
Comment 7 Ancor Gonzalez Sosa 2023-10-30 10:46:39 UTC
(In reply to Bret Towe from comment #0)
> 
> so the first item the installer needs to worry about is partitions need to
> allow setting an offset at the beginning

Can't you do that in the Expert Partitioner by selecting "Custom Region" while defining the partition size?
Comment 8 Ancor Gonzalez Sosa 2023-10-30 11:04:51 UTC
(In reply to Ancor Gonzalez Sosa from comment #6)
> (In reply to Bret Towe from comment #2)
> > 
> > it's a feature request to allow better support for any device that can't
> > have root os on the boot device for space (or performance) reasons.
> > as the installer does not allow for snapshots on /boot if its not on the
> > same device as the root os so I believe this problem could be hit on x86
> > systems also its just its a rare thing to do in that space
> 
> As far as I understand, the installer enforces /boot to be part of root
> file-system when snapshots are used

Sorry for adding confusion. My previous statement is is actually not precise. The installer actually allows to setup snapshots for the root file-system even if /boot is a separate file-system.

The setup that, indeed, the installer does not offer is having separate file-systems for /boot and / and to setup snapshots for BOTH file-systems. The installer only allows to setup snapshots for the root file-system, not any other.

> because that's a requisite for the standard snapshots tools we ship
> in the distribution.

That part of my previous message is still true. The reason for supporting only one kind of scenarios (snapshots enabled for / and not for other file-systems) is to match the expectation and default configuration of all the snapshots tools in the distribution.
Comment 9 Michal Suchanek 2023-10-30 11:16:39 UTC
I don't think it's possible to have snapshots span multiple filesystems.

As for original problem: its should be possible

1) install system into SD/emmc and use NVMe for data

2) build u-boot with NVMe support, install only u-boot to SD/emmc and boot system form NVMe

Some pine64 hardware also may have a SPI flash memory for installing u-boot. When u-boot is installed there the generic arm64 EFI image can be used on any medium u-boot supports.
Comment 10 Michal Suchanek 2023-10-30 11:22:21 UTC
What's the status of RockChip NVMe support in u-boot?

Last time I looked into it there was some downstream u-boot build with some extra patches to support his.
Comment 11 Bret Towe 2023-10-30 14:48:19 UTC
(In reply to Ancor Gonzalez Sosa from comment #7)
> (In reply to Bret Towe from comment #0)
> > 
> > so the first item the installer needs to worry about is partitions need to
> > allow setting an offset at the beginning
> 
> Can't you do that in the Expert Partitioner by selecting "Custom Region"
> while defining the partition size?

I don't recall seeing this, I only recall being able to set max size not start point
Comment 12 Bret Towe 2023-10-30 14:51:42 UTC
(In reply to Michal Suchanek from comment #10)
> What's the status of RockChip NVMe support in u-boot?
> 
> Last time I looked into it there was some downstream u-boot build with some
> extra patches to support his.

its unsupported in the kwiboo git branch at this time. once the patches are usable kwiboo tends add them so I assume whatever patches exist at the moment are not stable.
so it likely will work 'at some point' but at least in short term it just not a choice
Comment 13 Ancor Gonzalez Sosa 2023-10-30 15:14:44 UTC
Created attachment 870518 [details]
Step to add a partition

(In reply to Bret Towe from comment #11)
> 
> I don't recall seeing this, I only recall being able to set max size not
> start point

The option is there for every new partition you create with Expert Partition. See attachment.
Comment 14 Bret Towe 2023-10-30 16:26:27 UTC
(In reply to Ancor Gonzalez Sosa from comment #13)
> Created attachment 870518 [details]
> Step to add a partition
> 
> (In reply to Bret Towe from comment #11)
> > 
> > I don't recall seeing this, I only recall being able to set max size not
> > start point
> 
> The option is there for every new partition you create with Expert
> Partition. See attachment.

so it does. any chance to have an option to choose by megabyte or the likes instead?