Bug 1212376

Summary: Inconsistencies seen when the build system is checking the installation of systemd packages with file-triggers
Product: [openSUSE] openSUSE Tumbleweed Reporter: Franck Bui <fbui>
Component: BasesystemAssignee: systemd maintainers <systemd-maintainers>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: fvogt
Version: Current   
Target Milestone: ---   
Hardware: Other   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Franck Bui 2023-06-14 15:50:59 UTC
When building the latest version of systemd with file-triggers support added, the logs show the following messages during the verification process of the freshly built packages by OBS:

> [   93s] Verifying packages...                                                                                                                                                                                                                                                           
> [   93s] Preparing packages...                                                                                                                                                                                                                                                           
> [   93s] libudev1-253.5-1.41.x86_64                                                                                                                                                                                                                                                      
> [   93s] libsystemd0-253.5-1.41.x86_64                                                                                                                                                                                                                                                   
> [   93s] systemd-doc-253.5-1.41.x86_64                                                                                                                                                                                                                                                   
> [   93s] systemd-253.5-1.41.x86_64                                                                                                                                                                                                                                                       
> [   93s] warning: group systemd-journal does not exist - using root                                                                                                                                                                                                                      
> [   93s] Running in chroot, ignoring command 'daemon-reexec'                                                                                                                                                                                                                             
> [   93s] systemd-container-253.5-1.41.x86_64                                                                                                                                                                                                                                             
> [   94s] systemd-network-253.5-1.41.x86_64                                                                                                                                                                                                                                               
> [   94s] Creating group 'systemd-resolve' with GID 483.                                                                                                                                                                                                                                  
> [   94s] Creating user 'systemd-resolve' (systemd Resolver) with UID 483 and GID 483.                                                                                                                                                                                                    
> [   94s] systemd-experimental-253.5-1.41.x86_64                                                                                                                                                                                                                                          
> [   94s] Creating group 'systemd-oom' with GID 482.                                                                                                                                                                                                                                      
> [   94s] Creating user 'systemd-oom' (systemd Userspace OOM Killer) with UID 482 and GID 482.                                                                                                                                                                                            
> [   94s] systemd-journal-remote-253.5-1.41.x86_64                                                                                                                                                                                                                                        
> [   94s] Creating group 'systemd-journal-remote' with GID 481.                                                                                                                                                                                                                           
> [   94s] Creating user 'systemd-journal-remote' (systemd Journal Remote) with UID 481 and GID 481.                                                                                                                                                                                       
> [   94s] systemd-coredump-253.5-1.41.x86_64                                                                                                                                                                                                                                              
> [   94s] Creating group 'systemd-coredump' with GID 480.                                                                                                                                                                                                                                 
> [   94s] Creating user 'systemd-coredump' (systemd Core Dumper) with UID 480 and GID 480.                                                                                                                                                                                                
> [   94s] systemd-portable-253.5-1.41.x86_64                                                                                                                                                                                                                                              
> [   94s] systemd-testsuite-253.5-1.41.x86_64                                                                                                                                                                                                                                             
> [   94s] systemd-lang-253.5-1.41.noarch                                                                                                                                                                                                                                                  
> [   94s] udev-253.5-1.41.x86_64                                                                                                                                                                                                                                                          
> [   95s] Created symlink /etc/systemd/system/sysinit.target.wants/systemd-pstore.service -> /usr/lib/systemd/system/systemd-pstore.service.                                                                                                                                              
> [   95s] systemd-sysvcompat-253.5-1.41.x86_64                                                                                                                                                                                                                                            
> [   95s] systemd-devel-253.5-1.41.x86_64                                                                                                                                                                                                                                                 
> [   95s] systemd-devel-253.5-1.1.x86_64                                                                                                                                                                                                                                                  
> [   95s] libsystemd0-253.5-1.1.x86_64                                                                                                                                                                                                                                                    
> [   95s] libudev1-253.5-1.1.x86_64                                                                                                                                                                                                                                                       
> [   95s] regenerate-initrd-posttrans: dracut is not installed, not rebuilding the initrd                                                                                                                                                                                                 
> [   95s] /usr/lib/tmpfiles.d/journal-nocow.conf:26: Failed to resolve specifier: uninitialized /etc/ detected, skipping.                                                                                                                                                                 
> [   95s] All rules containing unresolvable specifiers will be skipped.                                                                                                                                                                                                                   
> [   95s] /usr/lib/tmpfiles.d/static-nodes-permissions.conf:17: Failed to resolve group 'kvm'.                                                                                                                                                                                            
> [   95s] /usr/lib/tmpfiles.d/static-nodes-permissions.conf:18: Failed to resolve group 'kvm'.                                                                                                                                                                                            
> [   95s] /usr/lib/tmpfiles.d/static-nodes-permissions.conf:19: Failed to resolve group 'kvm'.                                                                                                                                                                                            
> [   95s] /usr/lib/tmpfiles.d/systemd-network.conf:10: Failed to resolve user 'systemd-network': No such process                                                                                                                                                                          
> [   95s] /usr/lib/tmpfiles.d/systemd-network.conf:11: Failed to resolve user 'systemd-network': No such process                                                                                                                                                                          
> [   95s] /usr/lib/tmpfiles.d/systemd-network.conf:12: Failed to resolve user 'systemd-network': No such process                                                                                                                                                                          
> [   95s] /usr/lib/tmpfiles.d/systemd-network.conf:13: Failed to resolve user 'systemd-network': No such process                                                                                                                                                                          
> [   95s] /usr/lib/tmpfiles.d/systemd.conf:22: Failed to resolve group 'systemd-journal'.                                                                                                                                                                                                 
> [   95s] /usr/lib/tmpfiles.d/systemd.conf:25: Failed to resolve group 'systemd-journal'.                                                                                                                                                                                                 
> [   95s] error: lua script failed: [string "%transfiletriggerin(systemd-253.5-1.41.x86_64..."]:4: Unknown error 16640                                                                                                                                                                    
> [   95s] Creating group 'systemd-journal' with GID 479.                                                                                                                                                                                                                                  
> [   95s] Creating group 'systemd-network' with GID 478.                                                                                                                                                                                                                                  
> [   95s] Creating user 'systemd-network' (systemd Network Management) with UID 478 and GID 478.                                                                                                                                                                                          
> [   95s] Creating group 'systemd-timesync' with GID 477.                                                                                                                                                                                                                                 
> [   95s] Creating user 'systemd-timesync' (systemd Time Synchronization) with UID 477 and GID 477.

It seems that the creation of system users (done by systemd-sysusers) is intermixed with the creation of the tmpfiles (done by systemd-tmpfiles). And this is not working well because systemd-tmpfiles tries to set the ownership of some files to system users that have not been created yet by systemd-sysusers. This is the case for user "systemd-journal" for example.

And this is not expected: according to the comments the %transfiletriggerin scripts the priorities is such that systemd-sysusers should be called before systemd-tmpfiles is:

  %transfiletriggerin -P 100700 -p <lua> -- /usr/lib/sysusers.d                                                                                
  -- This script will process files installed in /usr/lib/sysusers.d to create                                                                 
  -- specified users automatically. The priority is set such that it                                                                           
  -- will run before the tmpfiles file trigger.                                                                                                
  assert(rpm.execute("systemd-sysusers"))

  %transfiletriggerin -P 1000600 -p <lua> -- /usr/lib/tmpfiles.d
  -- This script will process files installed in /usr/lib/tmpfiles.d to create
  -- tmpfiles automatically. The priority is set such that it will run
  -- after the sysusers file trigger, but before any other triggers.
  assert(rpm.execute("systemd-tmpfiles", "--create"))

The full build logs can be found at: https://build.opensuse.org/build/Base:System/standard/x86_64/systemd/_log and the implementation of the file-triggers for systemd is at https://build.opensuse.org/package/view_file/Base:System/systemd/triggers.systemd?expand=1
Comment 1 Franck Bui 2023-06-14 15:52:21 UTC
Michael, not sure if you are the good candidate, if not please re-assign it accordingly.
Comment 2 Fabian Vogt 2023-06-21 06:55:19 UTC
Let me remove some lines, then the cause should be visible:

  %transfiletriggerin -P 100700 -p <lua> -- /usr/lib/sysusers.d                                                                                
  %transfiletriggerin -P 1000600 -p <lua> -- /usr/lib/tmpfiles.d
Comment 3 Franck Bui 2023-06-21 08:45:46 UTC
Ouch, thanks Fabian for spotting, I blindly trusted upstream without even checking the basics :(

Let's keep this one opened until it's fixed in both upstream and Factory but let's also re-assign it accordingly.
Comment 4 Franck Bui 2023-06-21 08:46:52 UTC
FTR, fix submitted to upstream: https://github.com/systemd/systemd/pull/28102
Comment 5 OBSbugzilla Bot 2023-06-21 13:35:03 UTC
This is an autogenerated message for OBS integration:
This bug (1212376) was mentioned in
https://build.opensuse.org/request/show/1094372 Factory / systemd
Comment 6 Franck Bui 2023-06-22 09:15:10 UTC
Fix submitted to Factory and accepted by upstream, hence closing.