Bug 1173551 - OBS-built modules (like zfs) are signed by an untrusted key
OBS-built modules (like zfs) are signed by an untrusted key
Status: NEW
Classification: openSUSE
Product: openSUSE Distribution
Classification: openSUSE
Component: Basesystem
Leap 15.2
Other Other
: P5 - None : Normal (vote)
: ---
Assigned To: openSUSE Kernel Bugs
E-mail List
Depends on:
  Show dependency treegraph
Reported: 2020-07-01 00:55 UTC by Aleksa Sarai
Modified: 2021-01-01 22:33 UTC (History)
3 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Note You need to log in before you can comment on or make changes to this bug.
Description Aleksa Sarai 2020-07-01 00:55:17 UTC
Leap 15.2 blocks loading of modules which are not signed by a trusted key (which is an entirely reasonable policy)[1], however this means that if you wish to use modules which are built and signed by OBS it is now much more complicated.

Unlike modules compiled locally, OBS manages the signing keys (and rotates them fairly frequently) meaning that your system may end up failing to boot cleanly after a routine system update. In addition, you'd need to do the following in order to fix the issue:

 % osc signkey <project> --sslcert > cert.pem
 % openssl x509 -inform PEM -in cert.pem -outform DER cert.der
 % mokutil --import cert.der

And then reboot and enroll the key. In fairness this is technically documented in [2] but I doubt most users would be able to get to that point (not to mention I think OBS has changed their interface so downloading the project signing key is a little more hidden now).

Is it possible for this process to be more automated -- ideally such that key rotations are handled automatically by either zypper (since this is a problem that affects all kernel modules signed by OBS) or by the individual packages?

For a concrete example of this problem, I just upgraded from Leap 15.1 to the 15.2 beta and my system was fairly broken because ZFS wouldn't load (as well as wireguard -- though luckily that is in the Leap kernel package now). I've imported the filesystems key into the MOK, but that key expires in November 2020 and I guarantee I'm going to forget about this hack and will need to re-troubleshoot why my box was broken after a reboot.

[1]: https://doc.opensuse.org/release-notes/x86_64/openSUSE/Leap/15.2/#sec.driver.sig
[2]: https://en.opensuse.org/openSUSE:UEFI#Booting_a_kernel_from_Kernel:stable_repository
Comment 1 Malte Gell 2020-11-23 20:03:34 UTC
I usually build my own kernel, based either on the standard Leap kernel sources or from Kernel:stable

In both cases, I always use my own key for kernel signing and module signing. This way I do not have to deal with frequently changing keys. I have one key, my own key.

But, this requires to build the kernel on my own, every time the kernel gets upgraded. It´s not a big deal to me.
Comment 2 Aleksa Sarai 2020-11-24 08:06:06 UTC
It seems to me that a simpler solution would be to ship unsigned modules from third-party repositories, given that openSUSE/OBS cannot currently deal with this scenario in a reasonable way (I understand why we can't auto-sign modules from other OBS repositories but it seems odd to not make exceptions for well-maintained projects like filesystems).

Anyway, I just rotated the trusted on my server (which involves wheeling it across the room so it can have a keyboard and monitor attached), so see you again in 2023 when I have to do this again.
Comment 3 Takashi Iwai 2020-11-24 08:23:01 UTC
I investigated the issue recently.  The rpm macro for building a KMP has already an option to create a cert subpackage, and each KMP may use it.

In a nutshell, you need three things for a KMP:

1. Add '# needssslcertforbuild' line at the beginning of the spec file

2. Pass -c %{sourcedir}/_projectcert.crt option to
   kernel_module_package macro, e.g. 
     %kernel_module_package -p preamble -c %{sourcedir}/_projectcert.crt

3. Add the line below in %install section
     export BRP_PESIGN_FILES="*.ko"

(1) imports the _project.crt file from the OBS project, (2) enables the $PKG-ueficert subpackage creation, and (3) enables the repackaging for signing modules.  Many KMPs already have (1) and (3) but no (2).

The *-ueficert package installs/deletes the cert file and invokes mokutil just like the kernel package.

The obvious problem is, so far, that the procedure above isn't well documented, and the actual usage isn't quite user-friendly.  User has to install the *-ueficert package manually if they know it's really needed, and handle MOK deployment later.  We definitely need improvement in documentation...

Also the cert subpackage is basically superfluous for the KMPs that are shipped with the official cert.  So it'd be needed only for downstream KMPs like zfs.