Bug 1224149

Summary: [SELinux] sdbootutil (snapperd_t) fails to execute systemd-pcrlock (init_exec_t)
Product: [openSUSE] openSUSE Tumbleweed Reporter: Andrei Borzenkov <arvidjaar>
Component: SecurityAssignee: Zdenek Kubala <zkubala>
Status: CONFIRMED --- QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: aplanas, arvidjaar, gayane.osipyan, zkubala
Version: CurrentFlags: zkubala: needinfo? (arvidjaar)
Target Milestone: ---   
Hardware: Other   
OS: Other   
See Also: https://bugzilla.suse.com/show_bug.cgi?id=1224120
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Andrei Borzenkov 2024-05-12 06:01:20 UTC
Operating System: openSUSE MicroOS
10:~ # sestatus
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Memory protection checking:     actual (secure)
Max kernel policy version:      33
10:~ # zypper info selinux-policy
Loading repository data...
Reading installed packages...


Information for package selinux-policy:
---------------------------------------
Repository     : openSUSE-Tumbleweed-Oss
Name           : selinux-policy
Version        : 20240321-1.2
Arch           : noarch
Vendor         : openSUSE
Installed Size : 24.8 KiB
Installed      : Yes (automatically)
Status         : up-to-date
Source package : selinux-policy-20240321-1.2.src
Upstream URL   : https://github.com/fedora-selinux/selinux-policy.git
Summary        : SELinux policy configuration
Description    : 
    SELinux Reference Policy. A complete SELinux policy that can be used
    as the system policy for a variety of systems and used as the basis for
    creating other policies.

10:~ # rpm -q sdbootutil
sdbootutil-1+git20240506.573a6a4-1.1.x86_64
10:~ # rpm -qf /usr/lib/systemd/systemd-pcrlock 
systemd-experimental-255.4-3.1.x86_64
10:~ # ls -l /etc/systemd/tpm2-pcr-public-key.pem /etc/systemd/tpm2-pcr-private-key.pem
ls: cannot access '/etc/systemd/tpm2-pcr-public-key.pem': No such file or directory
ls: cannot access '/etc/systemd/tpm2-pcr-private-key.pem': No such file or directory
10:~ # 

sdbootutil defaults to systemd-pcrlock if is is present and no previous keypair for the signed policy is present.

10:~ # semodule -DB
10:~ # systemctl start snapper-cleanup.service
10:~ # semodule -B
10:~ # ausearch -m AVC,USER_AVC,SELINUX_ERR,USER_SELINUX_ERR -ts boot 
----
time->Sun May 12 08:54:27 2024
type=AVC msg=audit(1715493267.232:132): avc:  denied  { execute } for  pid=1325 comm="sdbootutil" name="systemd-pcrlock" dev="dm-0" ino=57169 scontext=system_u:system_r:snapperd_t:s0 tcontext=system_u:object_r:init_exec_t:s0 tclass=file permissive=0
----
time->Sun May 12 08:54:27 2024
type=AVC msg=audit(1715493267.232:133): avc:  denied  { execute } for  pid=1325 comm="sdbootutil" name="systemd-pcrlock" dev="dm-0" ino=57169 scontext=system_u:system_r:snapperd_t:s0 tcontext=system_u:object_r:init_exec_t:s0 tclass=file permissive=0
10:~ # 

Other denials for snapper_t are list in boo#1224120 for which I have local policy override.
Comment 1 Andrei Borzenkov 2024-05-19 04:53:53 UTC
After adding override for the reported AVCs I then got

10:~ # semodule -DB
10:~ # ausearch -m AVC,USER_AVC,SELINUX_ERR,USER_SELINUX_ERR -ts boot 
<no matches>
10:~ # systemctl start snapper-cleanup.service
10:~ # ausearch -m AVC,USER_AVC,SELINUX_ERR,USER_SELINUX_ERR -ts boot 
----
time->Sun May 19 07:50:37 2024
type=AVC msg=audit(1716094237.445:227): avc:  denied  { execute_no_trans } for  pid=1364 comm="sdbootutil" path="/usr/lib/systemd/systemd-pcrlock" dev="dm-0" ino=63823 scontext=system_u:system_r:snapperd_t:s0 tcontext=system_u:object_r:init_exec_t:s0 tclass=file permissive=0
Comment 2 Gayane Osipyan 2024-05-27 08:30:04 UTC
Hi ,

I have latest microos setup and not able reproduce this issue.here are version i have running.

NAME="openSUSE MicroOS"
# VERSION="20240523"
rpm -q sdbootutil
sdbootutil-1+git20240514.56dc89c-54.1.x86_64
rpm -qf /usr/lib/systemd/systemd-pcrlock 
systemd-experimental-255.6-2.1.x86_64

ausearch -m AVC,USER_AVC,SELINUX_ERR,USER_SELINUX_ERR
<no matches>
Comment 3 Andrei Borzenkov 2024-05-27 08:48:10 UTC
(In reply to Gayane Osipyan from comment #2)
> Hi ,
> 
> I have latest microos setup and not able reproduce this issue.

Yes, they are marked as dontaudit normally, so I used

semodule -DB

to catch them.
Comment 4 Gayane Osipyan 2024-05-27 08:58:52 UTC
(In reply to Andrei Borzenkov from comment #3)
> (In reply to Gayane Osipyan from comment #2)
> > Hi ,
> > 
> > I have latest microos setup and not able reproduce this issue.
> 
> Yes, they are marked as dontaudit normally, so I used
> 
> semodule -DB
> 
> to catch them.

Thanks for reply.
I did follow all steps you described and still got nothing ,is there something else need to be done to trigger those AVC's ?
Comment 5 Andrei Borzenkov 2024-05-31 17:56:49 UTC
(In reply to Gayane Osipyan from comment #4)

> I did follow all steps you described and still got nothing ,is there
> something else need to be done to trigger those AVC's ?

I assume you need to have snapshots eligible for removal. Default is

bor@10:~> sudo snapper get-config | grep NUMBER
NUMBER_CLEANUP         │ yes
NUMBER_LIMIT           │ 2-10
NUMBER_LIMIT_IMPORTANT │ 4-10
NUMBER_MIN_AGE         │ 1800
bor@10:~> 

So you need to have more than 10 snapshots and the oldest must be created at least 3 hours ago. I have more than 10 snashots and I always did "zypper dup" and reboot before testing so I always had extra snapshots. I suppose you could reduce them to the minimum, like

snapper set-config NUMBER_LIMIT=1 NUMBER_LIMIT_IMPORTANT=1

transactional-update shell
exit
reboot

transactional-update shell
exit
reboot

transactional-update cleanup

now you should have at least 2 extra snapshots and snapper-cleanup.service should do something.

Instead of "transactional-update shell" you can run any transactional-update command that creates new snapshot, like dup, pkg install or similar.
Comment 6 Alberto Planas Dominguez 2024-06-05 18:25:10 UTC
I reproduced this issue. The problem is this: sdbootutil is adding a new snapper plugin in "/usr/lib/snapper/plugins/10-sdbootutil.snapper". This is a bash script that will be called by snapper when a new snapsot is created or removed, for example.

When a snapshot gets removed, the plugin this calls '/usr/bin/sdbootutil remove-all-kernels "$num"', to clean the kernel and the boot loader entries from the image.

If I execute the command from outside snapper, this works and the entry gets removed, but when is snapper the one that calls it, selinux complains.

To fully reproduce it from the sdboot image that is in https://build.opensuse.org/package/show/devel:microos:images/openSUSE-MicroOS

# After installation, install any package
transactional-update pkg in emacs-nox

# Lets assume that this created a new transaction with id "2"
snapper ls

# The new transaction should be the default, we switch back to "1" as default
btrfs subvolume list -o /.snapshots
btrfs subvolume set-default 258 /.snapshots

# Now we can remove the snapshot
snapper rm 2

This last command will trigger the remove-all-kernels, that fails with:

bootctl unlink opensuse-microos-6.9.3-1-default-2.conf
opensuse-microos-6.9.3-1-default-2.conf is the default boot entry
Failed to remove "/boot/efi/loader/entries/opensuse-microos-6.9.3-1-default-2.conf": Permission denied

You can see in the selinux side this:

semodule -DB
ausearch -m AVC,USER_AVC,SELINUX_ERR,USER_SELINUX_ERR -ts boot

----
time->Wed Jun  5 20:04:56 2024
type=AVC msg=audit(1717610696.275:408): avc:  denied  { unlink } for  pid=9576 comm="bootctl" name="opensuse-microos-6.9.3-1-default-2.conf" dev="vda2" ino=59 scontext=system_u:system_r:snapperd_t:s0 tcontext=system_u:object_r:dosfs_t:s0 tclass=file permissive=0


So snapper should have permissions to access bootctl or something like that
Comment 7 Andrei Borzenkov 2024-06-05 19:01:02 UTC
(In reply to Alberto Planas Dominguez from comment #6)
> 
> So snapper should have permissions to access bootctl or something like that

I have these overrides that eliminate all denials on MicroOS systemd-boot image. Not sure how secure they are.

#============= snapperd_t ==============

allow snapperd_t dosfs_t:file unlink;
allow snapperd_t var_lib_t:file unlink;
allow snapperd_t init_exec_t:file { execute execute_no_trans };

#============= systemd_fstab_generator_t ==============

allow systemd_fstab_generator_t init_t:bpf { map_read map_write };

#============= systemd_gpt_generator_t ==============

allow systemd_gpt_generator_t init_t:bpf { map_read map_write };
Comment 8 Zdenek Kubala 2024-06-06 07:29:10 UTC
(In reply to Alberto Planas Dominguez from comment #6)
> I reproduced this issue. The problem is this: sdbootutil is adding a new
> snapper plugin in "/usr/lib/snapper/plugins/10-sdbootutil.snapper". This is
> a bash script that will be called by snapper when a new snapsot is created
> or removed, for example.
> 
> When a snapshot gets removed, the plugin this calls '/usr/bin/sdbootutil
> remove-all-kernels "$num"', to clean the kernel and the boot loader entries
> from the image.
> 
> If I execute the command from outside snapper, this works and the entry gets
> removed, but when is snapper the one that calls it, selinux complains.
> 
> To fully reproduce it from the sdboot image that is in
> https://build.opensuse.org/package/show/devel:microos:images/openSUSE-MicroOS
> 
> # After installation, install any package
> transactional-update pkg in emacs-nox
> 
> # Lets assume that this created a new transaction with id "2"
> snapper ls
> 
> # The new transaction should be the default, we switch back to "1" as default
> btrfs subvolume list -o /.snapshots
> btrfs subvolume set-default 258 /.snapshots
> 
> # Now we can remove the snapshot
> snapper rm 2
> 
> This last command will trigger the remove-all-kernels, that fails with:
> 
> bootctl unlink opensuse-microos-6.9.3-1-default-2.conf
> opensuse-microos-6.9.3-1-default-2.conf is the default boot entry
> Failed to remove
> "/boot/efi/loader/entries/opensuse-microos-6.9.3-1-default-2.conf":
> Permission denied
> 
> You can see in the selinux side this:
> 
> semodule -DB
> ausearch -m AVC,USER_AVC,SELINUX_ERR,USER_SELINUX_ERR -ts boot
> 
> ----
> time->Wed Jun  5 20:04:56 2024
> type=AVC msg=audit(1717610696.275:408): avc:  denied  { unlink } for 
> pid=9576 comm="bootctl" name="opensuse-microos-6.9.3-1-default-2.conf"
> dev="vda2" ino=59 scontext=system_u:system_r:snapperd_t:s0
> tcontext=system_u:object_r:dosfs_t:s0 tclass=file permissive=0
> 
> 
> So snapper should have permissions to access bootctl or something like that

This AVC denial has been firstly reported in bsc#1224120 and I have been looking into that and also it can be reproduced even without TPM2 `systemd-pcrlock` and FDE. So I would keep it separated and this unlink AVC handle in bsc#1224120.
Comment 9 Alberto Planas Dominguez 2024-06-06 07:38:00 UTC
(In reply to Zdenek Kubala from comment #8)
> (In reply to Alberto Planas Dominguez from comment #6)

> > So snapper should have permissions to access bootctl or something like that
> 
> This AVC denial has been firstly reported in bsc#1224120 and I have been
> looking into that and also it can be reproduced even without TPM2
> `systemd-pcrlock` and FDE. So I would keep it separated and this unlink AVC
> handle in bsc#1224120.

Would help if I copy the comment in the other bug?

Both bugs has the same root cause: the snapper plugin is calling sdbootutil in different stages: when the snapshot is created or deleted. In both cases bootctl is called and if FDE is set, pcr-oracle or systemd-pcrlock is also called to add or remove files.

Would be OK if all those fixes come together?
Comment 10 Zdenek Kubala 2024-06-10 13:49:23 UTC
I m taking over the bug and at the end  will fix this together with (bsc#1224120).
Comment 11 Zdenek Kubala 2024-06-28 11:40:49 UTC
(In reply to Andrei Borzenkov from comment #7)
> (In reply to Alberto Planas Dominguez from comment #6)
> > 
> > So snapper should have permissions to access bootctl or something like that
> 
> I have these overrides that eliminate all denials on MicroOS systemd-boot
> image. Not sure how secure they are.
> 
> #============= snapperd_t ==============
> 
> allow snapperd_t dosfs_t:file unlink;
> allow snapperd_t var_lib_t:file unlink;
> allow snapperd_t init_exec_t:file { execute execute_no_trans };
> 
> #============= systemd_fstab_generator_t ==============
> 
> allow systemd_fstab_generator_t init_t:bpf { map_read map_write };
> 
> #============= systemd_gpt_generator_t ==============
> 
> allow systemd_gpt_generator_t init_t:bpf { map_read map_write };


Hi Andrei, I'm not able to reproduce your errors. But could you please test this seliunx-policy packages from my OBS repository containing a fix for snapperd_t and dostfs_t (https://download.opensuse.org/repositories/home:/djz88:/branches:/security:/SELinux/openSUSE_Factory/)?

You can either add whole repo(1) or add specific packages(2) and install/update these two packages
- selinux-policy-targeted
- selinux-policy

(1) e.g.:
>zypper ar http://download.opensuse.org/repositories/home:/djz88:/branches:/security:/SELinux/openSUSE_Factory/home:djz88:branches:security:SELinux.repo

>zypper ref

>transactional-update pkg up selinux-policy=20240617-239.7 selinux-policy-targeted=20240617-239.7
>zypper se -s selinux-policy

(2) Or install/update directly 2 packages
>transactional-update pkg in https://download.opensuse.org/repositories/home:/djz88:/branches:/security:/SELinux/openSUSE_Factory/noarch/selinux-policy-20240617-239.7.noarch.rpm https://download.opensuse.org/repositories/home:/djz88:/branches:/security:/SELinux/openSUSE_Factory/noarch/selinux-policy-targeted-20240617-239.7.noarch.rpm

Of course before the updating package it would require to remove custom rules, which you put in place to fix locally the snapper issue.

BTW due to automatic rebuild maybe the version '-239.7' will change to '-239.XX'. Tthen just install the latest version in the repository(zypper se -s selinux-policy).

Thx