Bugzilla – Bug 1201066
kernel upgrade fails with Failed to enroll new keys
Last modified: 2023-10-05 16:29:40 UTC
Created attachment 859935 [details] /var/log/transactional-update.log MicroOS transactional-update fails on RPi. ># cat /sys/firmware/devicetree/base/model >Raspberry Pi 3 Model B Rev 1.2 > # head -2 /etc/os-release > NAME="openSUSE MicroOS" > # VERSION="20220622" > # uname -a > Linux wguard 5.18.4-1-default #1 SMP PREEMPT_DYNAMIC Wed Jun 15 06:00:33 UTC 2022 (ed6345d) aarch64 aarch64 aarch64 GNU/Linux ># transactional-update >... >The following product is going to be upgraded: >openSUSE MicroOS > 20220622-0 -> 20220628-0 >... >(25/37) Installing: kernel-default-5.18.6-1.1.aarch64 [.... >... >dracut: *** Creating image file '/boot/initrd-5.18.6-1-default' *** >dracut: *** Creating initramfs image file '/boot/initrd-5.18.6-1-default' done *** >Failed to enroll new keys >Failed to import /etc/uefi/certs/1F673297.crt >warning: %post(kernel-default-5.18.6-1.1.aarch64) scriptlet failed, exit status 255 >... >2022-06-30 17:21:02 Application returned with exit status 107. >2022-06-30 17:21 ERROR: zypper up on /.snapshots/3/snapshot failed with exit code 107!
Hmm, isn't shim updated together for the new cert?
The shim task is only to block the old key via a deny list. The enrollment should work though, although I am not 100% sure that this works on Arm64. would be nice to run this mokutil stand alone. try perhaps: mokutil --import /etc/uefi/certs/1F673297.crt --root-pw --ignore-keyring
(In reply to Marcus Meissner from comment #2) > try perhaps: > > mokutil --import /etc/uefi/certs/1F673297.crt --root-pw --ignore-keyring localhost:~ # mokutil --import /etc/uefi/certs/1F673297.crt --root-pw --ignore-keyring Failed to enroll new keys localhost:~ # echo $? 255 localhost:~ # mokutil --test-key /etc/uefi/certs/1F673297.crt /etc/uefi/certs/1F673297.crt is already in the built-in trusted keyring localhost:~ # mokutil --sb-state SecureBoot disabled Platform is in Setup Mode
Created attachment 860011 [details] strace output openat(AT_FDCWD, "/sys/firmware/efi/efivars/MokNew-605dab50-e046-4300-abb6-3dd810dd8b23", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/sys/firmware/efi/efivars/MokNew-605dab50-e046-4300-abb6-3dd810dd8b23", O_WRONLY|O_CREAT|O_EXCL, 0600) = -1 EROFS (Read-only file system) ioctl(-1, FS_IOC_SETFLAGS, [0]) = -1 EBADF (Bad file descriptor) write(2, "Failed to enroll new keys\n", 26) = 26 exit_group(-1) = ? +++ exited with 255 +++
Does it work if Secure Boot is enabled?
(In reply to Takashi Iwai from comment #5) > Does it work if Secure Boot is enabled? I'm not sure how to do that - I tried this: ># mokutil --enable-validation >password length: 8~16 >input password: >input password again: >Failed to request new MokSB state
This usually needs to be enabled in BIOS.
(In reply to Marcus Meissner from comment #7) > This usually needs to be enabled in BIOS. This is a Raspberry Pi (as stated above, it's a Raspberry Pi 3 Model B Rev 1.2). Neither does it support Secure Boot, nor does it have a BIOS. OpenSUSE starts here by first getting Uboot (a Bootloader) loaded by the firmware, Uboot then starts Grub2 and then Grub2 does its things.
(In reply to Kilian Hanich from comment #8) > (In reply to Marcus Meissner from comment #7) > > This usually needs to be enabled in BIOS. > > This is a Raspberry Pi (as stated above, it's a Raspberry Pi 3 Model B Rev > 1.2). > > Neither does it support Secure Boot, nor does it have a BIOS. > > OpenSUSE starts here by first getting Uboot (a Bootloader) loaded by the > firmware, Uboot then starts Grub2 and then Grub2 does its things. Correct. U-Boot does not support secure boot or EFI variables in general.
Ok, I think I have found the case: mokutil thinks it's an UEFI system. And I also found out, why it thinks that it's an UEFI system: efivarfs (findable under /sys/firmware/efi/efivars) gets mounted. Other OSs (Raspberry Pi OS, Manjaro) for the Raspberry Pi don't do that. Considering that it's not an UEFI-System, I don't think openSUSE should do this either. It also doesn't matter if it's normal Tumbleweed or microOS. But what I don't understand is, why does efivarfs get mounted?
(In reply to Kilian Hanich from comment #10) > Ok, I think I have found the case: mokutil thinks it's an UEFI system. Which is correct. > And I also found out, why it thinks that it's an UEFI system: efivarfs > (findable under /sys/firmware/efi/efivars) gets mounted. > > Other OSs (Raspberry Pi OS, Manjaro) for the Raspberry Pi don't do that. > > Considering that it's not an UEFI-System, I don't think openSUSE should do > this either. openSUSE on RPi boots using UEFI, provided by u-boot. > It also doesn't matter if it's normal Tumbleweed or microOS. > > But what I don't understand is, why does efivarfs get mounted?
OK, U-Boot follows the "Embedded Base Boot Requirements Specification - Release v1.0" and as such does not have a full blown UEFI implementation. This includes that "SetVariable" is not supported. efivarfs gets mounted because systemd does that. And as such mokutil thinks it is working with a full blown UEFI implementation. Now, I tried to just add "noefi" as a kernel parameter and I have not noticed any bugs. Because efivarfs doesn't get mounted, mokutil acts properly and as such transactional-update actually succeeds updating. So, does somebody know if anything else depends on being on an UEFI system?
*** Bug 1203087 has been marked as a duplicate of this bug. ***
(In reply to Kilian Hanich from comment #12) > OK, U-Boot follows the "Embedded Base Boot Requirements Specification - > Release v1.0" and as such does not have a full blown UEFI implementation. > This includes that "SetVariable" is not supported. > > efivarfs gets mounted because systemd does that. And as such mokutil thinks > it is working with a full blown UEFI implementation. > > Now, I tried to just add "noefi" as a kernel parameter and I have not > noticed any bugs. Because efivarfs doesn't get mounted, mokutil acts > properly and as such transactional-update actually succeeds updating. > > So, does somebody know if anything else depends on being on an UEFI system? There have following functions need efivarfs. All of them need SetVariable: - rtc-efi - efibootmgr - MOK - uefi capsule update For reading requirement, "mokutil --sb-state" needs efivarfs for detecting "secure boot" status. But I am not sure who used it.
A f88814cc25 patch be introduced to v5.8 kernel to set flag to read only if SetVariable not be supported by firmware: commit f88814cc2578c121e6edef686365036db72af0ed [v5.8-rc7~14] Author: Ard Biesheuvel <ardb@kernel.org> Date: Wed Jul 8 13:01:57 2020 +0300 efi/efivars: Expose RT service availability via efivars abstraction fs/efivarfs/super.c static int efivarfs_fill_super(struct super_block *sb, struct fs_context *fc) { ... if (!efivar_supports_writes()) sb->s_flags |= SB_RDONLY; ...
(In reply to Joey Lee from comment #14) > For reading requirement, "mokutil --sb-state" needs efivarfs for detecting > "secure boot" status. But I am not sure who used it. systemd automatically mounts efivarfs
IIRC, uboot can only support UEFI SecureBoot in a special manner, which needs to specifically install the PK, KEK and db via the u-boot shell. See: https://github.com/u-boot/u-boot/blob/v2022.04/doc/develop/uefi/uefi.rst#configuring-uefi-secure-boot BTW, MokUtil shouldn't be able to create any EFI variable to uboot because uboot has not yet supported SetVariable [EFI_RT_SUPPORTED_SET_VARIABLE, 0x0040] in its real-time services table. See: https://github.com/u-boot/u-boot/blob/v2022.04/lib/efi_loader/efi_runtime.c#L124
(In reply to Chester Lin from comment #17) > BTW, MokUtil shouldn't be able to create any EFI variable to uboot because > uboot has not yet supported SetVariable [EFI_RT_SUPPORTED_SET_VARIABLE, > 0x0040] in its real-time services table. See: > > https://github.com/u-boot/u-boot/blob/v2022.04/lib/efi_loader/efi_runtime. > c#L124 Well, mokutil can't, it errors out, but it thinks it can because as it seems the efivar library authors haven't thought about the possibility that efivarfs could be read-only and only check if it exists (this function gets called when the library gets initialized) which is then relied upon by mokutil. https://github.com/rhboot/efivar/blob/main/src/efivarfs.c#L67
(In reply to Joey Lee from comment #15) > A f88814cc25 patch be introduced to v5.8 kernel to set flag to read only if > SetVariable not be supported by firmware: > > commit f88814cc2578c121e6edef686365036db72af0ed [v5.8-rc7~14] > Author: Ard Biesheuvel <ardb@kernel.org> > Date: Wed Jul 8 13:01:57 2020 +0300 > > efi/efivars: Expose RT service availability via efivars abstraction > > > fs/efivarfs/super.c > static int efivarfs_fill_super(struct super_block *sb, struct fs_context *fc) > { > ... > if (!efivar_supports_writes()) > sb->s_flags |= SB_RDONLY; > ... Kernel detected the EFI runtime properties table, and set s_flags to read only. It can block the writing job from userland. But it doesn't expose "Read Only" property to userland. By the way, I was sent patch to upstream for creating efivars mount point when efivarfs is available: https://lore.kernel.org/lkml/20200924095157.GA304421@kroah.com/T/#m1442b0ab6d0928f0e56f9aa1aed3e23b9907d88d Follow Greg's suggestion, we use /proc/filesystems to detect the existence of efivarfs in userland.
How about change the kernel scripts? To avoid the "255" return value to expose: suse-module-tools/kernel-scriptlets/cert-script ... # XXX: Only call mokutil if UEFI and shim are used for cert in $certs; do cert="/etc/uefi/certs/${cert}.crt" run_mokutil --import "$cert" --root-pw ${MOK_ARGS} rc=$? if [ $rc != 0 ] ; then script_rc=$rc # hide 255 here? echo "Failed to import $cert" >&2 fi done
(In reply to Joey Lee from comment #19) > Kernel detected the EFI runtime properties table, and set s_flags to read > only. It can block the writing job from userland. But it doesn't expose > "Read Only" property to userland. Well, it is marked as "ro" under mount.
(In reply to Joey Lee from comment #20) > How about change the kernel scripts? To avoid the "255" return value to > expose: > > suse-module-tools/kernel-scriptlets/cert-script > ... > # XXX: Only call mokutil if UEFI and shim are used > for cert in $certs; do > cert="/etc/uefi/certs/${cert}.crt" > run_mokutil --import "$cert" --root-pw ${MOK_ARGS} > rc=$? > if [ $rc != 0 ] ; then > script_rc=$rc # hide 255 here? > echo "Failed to import $cert" >&2 > fi > done I'd do it like this: # XXX: Only call mokutil if UEFI and shim are used and if efivarfs is mounted readable mount | grep efivarfs | grep -E '\(.*,?ro,?.*\)' if [ $? == 0 ] ; then for cert in $certs; do cert="/etc/uefi/certs/${cert}.crt" run_mokutil --import "$cert" --root-pw ${MOK_ARGS} rc=$? if [ $rc != 0 ] ; then script_rc=$rc echo "Failed to import $cert" >&2 fi done fi
(In reply to Kilian Hanich from comment #22) > (In reply to Joey Lee from comment #20) > > How about change the kernel scripts? To avoid the "255" return value to > > expose: > > > > suse-module-tools/kernel-scriptlets/cert-script > > ... > > # XXX: Only call mokutil if UEFI and shim are used > > for cert in $certs; do > > cert="/etc/uefi/certs/${cert}.crt" > > run_mokutil --import "$cert" --root-pw ${MOK_ARGS} > > rc=$? > > if [ $rc != 0 ] ; then > > script_rc=$rc # hide 255 here? > > echo "Failed to import $cert" >&2 > > fi > > done > > I'd do it like this: > > # XXX: Only call mokutil if UEFI and shim are used and if efivarfs is > mounted readable > mount | grep efivarfs | grep -E '\(.*,?ro,?.*\)' > if [ $? == 0 ] ; then > for cert in $certs; do > cert="/etc/uefi/certs/${cert}.crt" > run_mokutil --import "$cert" --root-pw ${MOK_ARGS} > rc=$? > if [ $rc != 0 ] ; then > script_rc=$rc > echo "Failed to import $cert" >&2 > fi > done > fi I agree. Will you send the change to https://github.com/openSUSE/suse-module-tools.git ?
(In reply to Joey Lee from comment #23) > (In reply to Kilian Hanich from comment #22) > > (In reply to Joey Lee from comment #20) > > > How about change the kernel scripts? To avoid the "255" return value to > > > expose: > > > > > > suse-module-tools/kernel-scriptlets/cert-script > > > ... > > > # XXX: Only call mokutil if UEFI and shim are used > > > for cert in $certs; do > > > cert="/etc/uefi/certs/${cert}.crt" > > > run_mokutil --import "$cert" --root-pw ${MOK_ARGS} > > > rc=$? > > > if [ $rc != 0 ] ; then > > > script_rc=$rc # hide 255 here? > > > echo "Failed to import $cert" >&2 > > > fi > > > done > > > > I'd do it like this: > > > > # XXX: Only call mokutil if UEFI and shim are used and if efivarfs is > > mounted readable > > mount | grep efivarfs | grep -E '\(.*,?ro,?.*\)' > > if [ $? == 0 ] ; then > > for cert in $certs; do > > cert="/etc/uefi/certs/${cert}.crt" > > run_mokutil --import "$cert" --root-pw ${MOK_ARGS} > > rc=$? > > if [ $rc != 0 ] ; then > > script_rc=$rc > > echo "Failed to import $cert" >&2 > > fi > > done > > fi > > I agree. > > Will you send the change to > https://github.com/openSUSE/suse-module-tools.git ? Sure, but I will try this first locally if it works in practice.
Suggested an alternative fix in https://github.com/openSUSE/suse-module-tools/pull/66
Joey Lee, I tried the fix today again as discussed in the new pull request. Unlike before there is a shim update now too. While the pull request fixes the problem with that script, shim's post script has the same problem. I will submit a revision there.
This is an autogenerated message for OBS integration: This bug (1201066) was mentioned in https://build.opensuse.org/request/show/1003010 Factory / suse-module-tools
(In reply to Martin Wilck from comment #25) > Suggested an alternative fix in > https://github.com/openSUSE/suse-module-tools/pull/66 Thanks for Martin and Kilian's help. Martin's patches in suse-module-tools is good to me.
(In reply to Kilian Hanich from comment #26) > Joey Lee, I tried the fix today again as discussed in the new pull request. > > Unlike before there is a shim update now too. > > While the pull request fixes the problem with that script, shim's post > script has the same problem. > > I will submit a revision there. Thanks for your help! I just pushed your change to devel:openSUSE:Factory/shim.
I'm not really confident from the comments, so is this fixed? If not, feel free to reopen.
SUSE-SU-2023:1702-1: An update that solves one vulnerability, contains two features and has 10 fixes can now be installed. Category: security (important) Bug References: 1185232, 1185261, 1185441, 1185621, 1187071, 1187260, 1193282, 1198458, 1201066, 1202120, 1205588 CVE References: CVE-2022-28737 Jira References: PED-127, PED-1273 Sources used: openSUSE Leap Micro 5.3 (src): shim-15.7-150300.4.11.1 openSUSE Leap 15.4 (src): shim-15.7-150300.4.11.1 SUSE Linux Enterprise Micro for Rancher 5.3 (src): shim-15.7-150300.4.11.1 SUSE Linux Enterprise Micro 5.3 (src): shim-15.7-150300.4.11.1 SUSE Linux Enterprise Micro for Rancher 5.4 (src): shim-15.7-150300.4.11.1 SUSE Linux Enterprise Micro 5.4 (src): shim-15.7-150300.4.11.1 Basesystem Module 15-SP4 (src): shim-15.7-150300.4.11.1 SUSE Linux Enterprise High Performance Computing ESPOS 15 SP3 (src): shim-15.7-150300.4.11.1 SUSE Linux Enterprise High Performance Computing LTSS 15 SP3 (src): shim-15.7-150300.4.11.1 SUSE Linux Enterprise Real Time 15 SP3 (src): shim-15.7-150300.4.11.1 SUSE Linux Enterprise Server 15 SP3 LTSS 15-SP3 (src): shim-15.7-150300.4.11.1 SUSE Linux Enterprise Server for SAP Applications 15 SP3 (src): shim-15.7-150300.4.11.1 SUSE Manager Proxy 4.2 (src): shim-15.7-150300.4.11.1 SUSE Manager Retail Branch Server 4.2 (src): shim-15.7-150300.4.11.1 SUSE Manager Server 4.2 (src): shim-15.7-150300.4.11.1 SUSE Enterprise Storage 7.1 (src): shim-15.7-150300.4.11.1 SUSE Linux Enterprise Micro 5.1 (src): shim-15.7-150300.4.11.1 SUSE Linux Enterprise Micro 5.2 (src): shim-15.7-150300.4.11.1 SUSE Linux Enterprise Micro for Rancher 5.2 (src): shim-15.7-150300.4.11.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
SUSE-SU-2023:1863-1: An update that solves one vulnerability, contains two features and has 11 fixes can now be installed. Category: security (important) Bug References: 1185232, 1185261, 1185441, 1185621, 1187071, 1187260, 1193282, 1193315, 1198458, 1201066, 1202120, 1205588 CVE References: CVE-2022-28737 Jira References: PED-127, PED-1273 Sources used: SUSE OpenStack Cloud 9 (src): shim-15.7-25.24.1 SUSE OpenStack Cloud Crowbar 9 (src): shim-15.7-25.24.1 SUSE Linux Enterprise Server for SAP Applications 12 SP4 (src): shim-15.7-25.24.1 SUSE Linux Enterprise Server 12 SP4 ESPOS 12-SP4 (src): shim-15.7-25.24.1 SUSE Linux Enterprise Server 12 SP4 LTSS 12-SP4 (src): shim-15.7-25.24.1 SUSE Linux Enterprise Server for SAP Applications 12 SP5 (src): shim-15.7-25.24.1 SUSE Linux Enterprise High Performance Computing 12 SP5 (src): shim-15.7-25.24.1 SUSE Linux Enterprise Server 12 SP5 (src): shim-15.7-25.24.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
SUSE-SU-2023:2086-1: An update that solves one vulnerability, contains two features and has 12 fixes can now be installed. Category: security (important) Bug References: 1185232, 1185261, 1185441, 1185621, 1187071, 1187260, 1193282, 1193315, 1198101, 1198458, 1201066, 1202120, 1205588 CVE References: CVE-2022-28737 Jira References: PED-127, PED-1273 Sources used: SUSE Linux Enterprise High Performance Computing 15 SP1 LTSS 15-SP1 (src): shim-15.7-150100.3.35.1 SUSE Linux Enterprise High Performance Computing 15 SP2 LTSS 15-SP2 (src): shim-15.7-150100.3.35.1 SUSE Linux Enterprise Server 15 SP1 LTSS 15-SP1 (src): shim-15.7-150100.3.35.1 SUSE Linux Enterprise Server 15 SP2 LTSS 15-SP2 (src): shim-15.7-150100.3.35.1 SUSE Linux Enterprise Server for SAP Applications 15 SP1 (src): shim-15.7-150100.3.35.1 SUSE Linux Enterprise Server for SAP Applications 15 SP2 (src): shim-15.7-150100.3.35.1 SUSE Enterprise Storage 7 (src): shim-15.7-150100.3.35.1 SUSE CaaS Platform 4.0 (src): shim-15.7-150100.3.35.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
SUSE-SU-2023:2150-1: An update that solves one vulnerability, contains two features and has 11 fixes can now be installed. Category: security (important) Bug References: 1185232, 1185261, 1185441, 1185621, 1187071, 1187260, 1187696, 1193282, 1198458, 1201066, 1202120, 1205588 CVE References: CVE-2022-28737 Jira References: PED-127, PED-1273 Sources used: SUSE Linux Enterprise Server 12 SP2 BCL 12-SP2 (src): shim-15.7-22.15.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
SUSE-RU-2023:3986-1: An update that has two fixes can now be installed. Category: recommended (important) Bug References: 1201066, 1213428 Sources used: openSUSE Leap 15.4 (src): suse-module-tools-15.4.17-150400.3.11.1 SUSE Linux Enterprise Micro for Rancher 5.3 (src): suse-module-tools-15.4.17-150400.3.11.1 SUSE Linux Enterprise Micro 5.3 (src): suse-module-tools-15.4.17-150400.3.11.1 SUSE Linux Enterprise Micro for Rancher 5.4 (src): suse-module-tools-15.4.17-150400.3.11.1 SUSE Linux Enterprise Micro 5.4 (src): suse-module-tools-15.4.17-150400.3.11.1 Basesystem Module 15-SP4 (src): suse-module-tools-15.4.17-150400.3.11.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
SUSE-RU-2023:3985-1: An update that has four fixes can now be installed. Category: recommended (important) Bug References: 1201066, 1212957, 1213428, 1213822 Sources used: openSUSE Leap 15.5 (src): suse-module-tools-15.5.2-150500.3.3.1 Basesystem Module 15-SP5 (src): suse-module-tools-15.5.2-150500.3.3.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.