Bug 1211849 - error: lua script failed: [string "%transfiletriggerin(systemd-253.4-2.1.x86_64)..."]:4: Unknown error 16640
Summary: error: lua script failed: [string "%transfiletriggerin(systemd-253.4-2.1.x86_...
Status: RESOLVED FIXED
Alias: None
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Basesystem (show other bugs)
Version: Current
Hardware: Other Other
: P2 - High : Major (vote)
Target Milestone: ---
Assignee: Joey Lee
QA Contact: E-mail List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-05-31 08:29 UTC by Franck Bui
Modified: 2023-06-26 10:21 UTC (History)
4 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---
fbui: needinfo? (mls)


Attachments
File trigger support for the repackage script (4.31 KB, patch)
2023-06-02 14:52 UTC, Michael Schröder
Details | Diff
fix-filetriggers.patch (4.43 KB, patch)
2023-06-09 14:33 UTC, Joey Lee
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Franck Bui 2023-05-31 08:29:28 UTC
systemd package recently gained support for file triggers. The implementation details can be found at https://build.opensuse.org/package/view_file/Base:System/systemd/triggers.systemd?expand=1.

My basic testing on my local builds showed that the basic functionalities (at least) were working as expected.

This was submitted to Factory a few weeks ago and has been released in TW with snapshot 20230524.

To my surprise, running on TW `rpm -q --filetriggers systemd` with the latest version of systemd doesn't show anything.

The problem is probably related to this error that can be found in the systemd build log[1]:

> [  753s] error: lua script failed: [string "%transfiletriggerin(systemd-253.4-2.1.x86_64)..."]:4: Unknown error 16640

For some reasons this errors is silently ignored, which is probably a bug per se.

Again, this problem seems specific to the build done by OBS as my local builds of systemd package don't show this error and `rpm -q --filetriggers` on them works as expected.

[1] https://build.opensuse.org/build/openSUSE:Factory/standard/x86_64/systemd/_log
Comment 1 Michael Schröder 2023-05-31 11:26:09 UTC
The packages build fine, it's the post-build installation check that displays this error. What happens is that a trigger is called and does a rpm.execute().
For some reason the executed program returns exit code 65 (waitpid status 16640).

Because of a programming mistake in rpm.execute this will result in a (nil, "Unknown error 16640", 16640) triplet. The assert statement in the scriptlet will then fail with the "Unknown error 16640" message that you see.
Comment 2 Michael Schröder 2023-05-31 11:36:44 UTC
Regarding the missing file triggers: the "pesign-repackage" script kills them in the repackage step!
Comment 3 Michael Schröder 2023-05-31 11:42:39 UTC
Reassigned to pesign-obs-integration maintainer.

Ceterum censeo: the repackage should be killed with fire! The signature files should be added without doing a rpm build. Everything else is way too error prone.
Comment 4 Franck Bui 2023-05-31 13:03:14 UTC
(In reply to Michael Schröder from comment #1)
> The packages build fine, it's the post-build installation check that
> displays this error. What happens is that a trigger is called and does a
> rpm.execute().
> For some reason the executed program returns exit code 65 (waitpid status
> 16640).

65 seems the error code returned by systemd-tmpfiles when some lines of the configuration files had to be ignored due to errors. The following errors taken from the build logs probably give some hints about the actual errors:

> [  753s] /usr/lib/tmpfiles.d/systemd-network.conf:13: Failed to resolve user 'systemd-network': No such process
> [  753s] /usr/lib/tmpfiles.d/systemd.conf:22: Failed to resolve group 'systemd-journal'.
> [...]
> [  753s] error: lua script failed: [string "%transfiletriggerin(systemd-253.4-2.1.x86_64)..."]:4: Unknown error 16640

Apparently sysuser/group 'systemd-network' and 'systemd-journal' are not yet created at this point.

The logs show that they have been created after:

> [  753s] Creating group 'systemd-journal' with GID 482.
> [  753s] Creating group 'systemd-network' with GID 481.

And this is not expected: according to the comments the %transfiletriggerin scripts the priorities is such that systemd-sysusers should be called before systemd-tmpfiles is:

  %transfiletriggerin -P 100700 -p <lua> -- /usr/lib/sysusers.d                                                                                
  -- This script will process files installed in /usr/lib/sysusers.d to create                                                                 
  -- specified users automatically. The priority is set such that it                                                                           
  -- will run before the tmpfiles file trigger.                                                                                                
  assert(rpm.execute("systemd-sysusers"))

  %transfiletriggerin -P 1000600 -p <lua> -- /usr/lib/tmpfiles.d
  -- This script will process files installed in /usr/lib/tmpfiles.d to create
  -- tmpfiles automatically. The priority is set such that it will run
  -- after the sysusers file trigger, but before any other triggers.
  assert(rpm.execute("systemd-tmpfiles", "--create"))

Do you know why these priorities are ignored in this case ?

> Because of a programming mistake in rpm.execute this will result in a (nil,
> "Unknown error 16640", 16640) triplet.

Did you mean that there's something wrong in:

   assert(rpm.execute("systemd-tmpfiles", "--create"))

If so can you show how this can be fixed ?
Comment 5 Gary Ching-Pang Lin 2023-06-01 03:12:28 UTC
The official maintainer of pesign-obs-integration is Joey now. Reassign to him.
On the other hand, it seems that systemd-boot is the only one package need pesign-obs-integration. Virtualbox singles out the kmp package with multibuild to avoid repackaging the main package. This may work around the error.
Comment 6 Joey Lee 2023-06-01 06:40:05 UTC
Changing the behavior of pesign-obs-integration will affect other KMP/EFI packages. I do not prefer the change the repacking behavior.

I agree Gary's suggestion in comment#5, could systemd package please separate *.efi file to a separate rpm, like virtualbox?

Reset assigner to default.
Comment 7 Franck Bui 2023-06-01 08:53:03 UTC
(In reply to Joey Lee from comment #6)
> Changing the behavior of pesign-obs-integration will affect other KMP/EFI
> packages. I do not prefer the change the repacking behavior.

Well that's a bug, it should be fixed instead of working around it and waiting for more people to be trapped IMHO.

> I agree Gary's suggestion in comment#5, could systemd package please
> separate *.efi file to a separate rpm, like virtualbox?

That's actually already the case. The signed binary is systemd-bootx64.efi and is shipped by 'udev' sub package. However the file triggers from systemd main package are (also) missing.

> Reset assigner to default.

I'm not sure to understand why. If your the maintainer of pesign-obs-integration you're supposed to fix the problem or decide to close it without addressing it (which I would disagree at that point) but resetting the assignee to default just means that you basically wait for "someone else " (ie no one) to address the problem, which won't happen.
Comment 9 Franck Bui 2023-06-02 08:42:44 UTC
Reassigning back to the official maintainer of pesign-obs-integration in the hope that we'll make some progress quickly.
Comment 10 Joey Lee 2023-06-02 09:56:10 UTC
(In reply to Franck Bui from comment #9)
> Reassigning back to the official maintainer of pesign-obs-integration in the
> hope that we'll make some progress quickly.

I am not good on systemd. What should I do for this issue?
Comment 11 Franck Bui 2023-06-02 10:14:49 UTC
(In reply to Joey Lee from comment #10)
> I am not good on systemd. What should I do for this issue?

Try to fix pesign-obs-integration so it doesn't kill silently any filetriggers that a package initially have ?

Or even better fix the whole design of pesign-obs-integration  as suggested in comment #3 since the current design is too error prone ?
Comment 12 Joey Lee 2023-06-02 13:25:42 UTC
(In reply to Franck Bui from comment #11)
> (In reply to Joey Lee from comment #10)
> > I am not good on systemd. What should I do for this issue?
> 
> Try to fix pesign-obs-integration so it doesn't kill silently any
> filetriggers that a package initially have ?
> 

Can *.efi file be moved to a separate RPM?

> Or even better fix the whole design of pesign-obs-integration  as suggested
> in comment #3 since the current design is too error prone ?

In comment#3, Michael suggested that attaching signature without rpm build. Do you have good idea to attach signature on original *.ko or *.efi files but doesn't need to rebuild RPM? 

Does there have good way to repackage RPM but still keep the filetriggers?
Comment 14 Michael Schröder 2023-06-02 13:38:09 UTC
Why does the it repackage rpms that do not include files that needed to be signed?
Comment 15 Michael Schröder 2023-06-02 13:44:12 UTC
Adding transfiletrigger support to pesign-gen-repackage-spec should be pretty straightforward, though.
Comment 16 Franck Bui 2023-06-02 13:49:28 UTC
(In reply to Joey Lee from comment #12)
> Can *.efi file be moved to a separate RPM?

See my previous comment: systemd-bootx86.efi (the file that is signed) is part of udev (sub)package though the file-triggers of systemd main package are also killed. So that wouldn't help.

> In comment#3, Michael suggested that attaching signature without rpm build.
> Do you have good idea to attach signature on original *.ko or *.efi files
> but doesn't need to rebuild RPM? 
> 
> Does there have good way to repackage RPM but still keep the filetriggers?

No idea, sorry, I'm not involved in these areas.

However if repackaging is judged awkward then another (naive) approach would be to let packages deal with the signatures themself in their %install scriptlet. Ideally a package installed at build time would provide the relevant tools to do that easily. But I have no idea how the signing process with OBS works actually so this is probably not possible.
Comment 18 Michael Schröder 2023-06-02 14:48:47 UTC
Regarding that weird error message: I've opened an rpm upstream issue: https://github.com/rpm-software-management/rpm/issues/2528
Comment 19 Michael Schröder 2023-06-02 14:52:13 UTC
Created attachment 867378 [details]
File trigger support for the repackage script

Here's a pretty much untested patch for the repackage script that can be used as a starting point.
Comment 20 Franck Bui 2023-06-02 15:21:07 UTC
(In reply to Michael Schröder from comment #18)
> Regarding that weird error message: I've opened an rpm upstream issue:
> https://github.com/rpm-software-management/rpm/issues/2528

Thanks.

However the root cause of this error message is due to the fact that the priorities of the %transfiletriggerin scripts (tmpfiles before sysusers) seems not be fulfilled.
Comment 21 Joey Lee 2023-06-07 10:57:44 UTC
Hi Michael,

(In reply to Michael Schröder from comment #19)
> Created attachment 867378 [details]
> File trigger support for the repackage script
> 
> Here's a pretty much untested patch for the repackage script that can be
> used as a starting point.

Thanks for your solution. I am testing it. I will update it to pesign-obs-integration github by your name after testing success.
Comment 22 Joey Lee 2023-06-08 09:34:52 UTC
(In reply to Joey Lee from comment #21)
> Hi Michael,
> 
> (In reply to Michael Schröder from comment #19)
> > Created attachment 867378 [details]
> > File trigger support for the repackage script
> > 
> > Here's a pretty much untested patch for the repackage script that can be
> > used as a starting point.
> 
> Thanks for your solution. I am testing it. I will update it to
> pesign-obs-integration github by your name after testing success.

It did not pass the post-build-checks on OBS:

https://build.opensuse.org/package/live_build_log/home:joeyli:branches:Base:System/systemd/standard/x86_64

[   92s] /.build_patchrpmcheck_scr: line 1: fg: no job control
[   92s] preinstall script of systemd-253.5-5.8.x86_64.rpm failed

I am looking at the problem.
Comment 23 Joey Lee 2023-06-09 14:33:03 UTC
Created attachment 867485 [details]
fix-filetriggers.patch

(In reply to Michael Schröder from comment #19)
> Created attachment 867378 [details]
> File trigger support for the repackage script
> 
> Here's a pretty much untested patch for the repackage script that can be
> used as a starting point.

This is a patch fixed the issue in the patch on comment#19. The original patch generated wrong %filetrigger* and %transfiletrigger* string. It generated redundant "trigger", e.g. %filetriggertriggerin or %transfiletriggertriggerin. So filetrigger scriptlet can not be attached when building RPM.
Comment 24 Joey Lee 2023-06-12 08:21:58 UTC
(In reply to Joey Lee from comment #23)
> Created attachment 867485 [details]
> fix-filetriggers.patch
> 
> (In reply to Michael Schröder from comment #19)
> > Created attachment 867378 [details]
> > File trigger support for the repackage script
> > 
> > Here's a pretty much untested patch for the repackage script that can be
> > used as a starting point.
> 
> This is a patch fixed the issue in the patch on comment#19. The original
> patch generated wrong %filetrigger* and %transfiletrigger* string. It
> generated redundant "trigger", e.g. %filetriggertriggerin or
> %transfiletriggertriggerin. So filetrigger scriptlet can not be attached
> when building RPM.

I have merged Michael's patch to openSUSE/pesign-obs-integration on github. And I also upgraded pesign-obs-integration to 10.2+git20230612.4699910 version which includes Michael's patch for supporting filetrigger and transfiletrigger.

It should works to add transfiletrigger scripts back to systemd RPMs.
Comment 25 Joey Lee 2023-06-14 06:23:22 UTC
(In reply to Joey Lee from comment #24)
> (In reply to Joey Lee from comment #23)
> > Created attachment 867485 [details]
> > fix-filetriggers.patch
> > 
> > (In reply to Michael Schröder from comment #19)
> > > Created attachment 867378 [details]
> > > File trigger support for the repackage script
> > > 
> > > Here's a pretty much untested patch for the repackage script that can be
> > > used as a starting point.
> > 
> > This is a patch fixed the issue in the patch on comment#19. The original
> > patch generated wrong %filetrigger* and %transfiletrigger* string. It
> > generated redundant "trigger", e.g. %filetriggertriggerin or
> > %transfiletriggertriggerin. So filetrigger scriptlet can not be attached
> > when building RPM.
> 
> I have merged Michael's patch to openSUSE/pesign-obs-integration on github.
> And I also upgraded pesign-obs-integration to 10.2+git20230612.4699910
> version which includes Michael's patch for supporting filetrigger and
> transfiletrigger.
> 
> It should works to add transfiletrigger scripts back to systemd RPMs.

The pesign-obs-integration be upgraded to 10.2+git20230612.4699910 on Base:System and openSUSE:Factory.

The systemd-253.5-1.41.x86_64.rpm from Base:System has filetriggers now.

Set this issue to FIXED.
Comment 26 Franck Bui 2023-06-14 12:55:53 UTC
(In reply to Joey Lee from comment #25)
> The systemd-253.5-1.41.x86_64.rpm from Base:System has filetriggers now.

Thanks.

There's still one issue that needs to be clarified (see comment #4) but I'll open a new bug report for that.
Comment 27 Franck Bui 2023-06-14 12:57:18 UTC
(In reply to Joey Lee from comment #25)
> The pesign-obs-integration be upgraded to 10.2+git20230612.4699910 on
> Base:System and openSUSE:Factory.

Can IBS also be fixed so ALP won't be hit by the same issue ?
Comment 28 Joey Lee 2023-06-17 07:46:17 UTC
Reopen this issue against ALP.(In reply to Franck Bui from comment #27)
> (In reply to Joey Lee from comment #25)
> > The pesign-obs-integration be upgraded to 10.2+git20230612.4699910 on
> > Base:System and openSUSE:Factory.
> 
> Can IBS also be fixed so ALP won't be hit by the same issue ?

Reopen this issue against ALP.
Comment 29 Joey Lee 2023-06-22 13:20:06 UTC
(In reply to Joey Lee from comment #28)
> Reopen this issue against ALP.(In reply to Franck Bui from comment #27)
> > (In reply to Joey Lee from comment #25)
> > > The pesign-obs-integration be upgraded to 10.2+git20230612.4699910 on
> > > Base:System and openSUSE:Factory.
> > 
> > Can IBS also be fixed so ALP won't be hit by the same issue ?
> 
> Reopen this issue against ALP.

I have submit 10.2+git20230612.4699910 pesign-obs-integration from openSUSE:Factory to SUSE:ALP:Workbench:1.0 on IBS.

submitreq is:

https://build.suse.de/request/show/301904
Comment 30 Franck Bui 2023-06-22 14:16:43 UTC
(In reply to Joey Lee from comment #29)
> I have submit 10.2+git20230612.4699910 pesign-obs-integration from
> openSUSE:Factory to SUSE:ALP:Workbench:1.0 on IBS.
> 

Thanks!
Comment 31 Joey Lee 2023-06-26 02:01:37 UTC
(In reply to Joey Lee from comment #29)
> (In reply to Joey Lee from comment #28)
> > Reopen this issue against ALP.(In reply to Franck Bui from comment #27)
> > > (In reply to Joey Lee from comment #25)
> > > > The pesign-obs-integration be upgraded to 10.2+git20230612.4699910 on
> > > > Base:System and openSUSE:Factory.
> > > 
> > > Can IBS also be fixed so ALP won't be hit by the same issue ?
> > 
> > Reopen this issue against ALP.
> 
> I have submit 10.2+git20230612.4699910 pesign-obs-integration from
> openSUSE:Factory to SUSE:ALP:Workbench:1.0 on IBS.
> 
> submitreq is:
> 
> https://build.suse.de/request/show/301904

Re-submitreq to SUSE:ALP:Source:Standard:1.0 because SUSE:ALP:Workbench:1.0 is frozen:

https://build.suse.de/request/show/302111
Comment 32 Joey Lee 2023-06-26 10:21:47 UTC
(In reply to Joey Lee from comment #31)
> (In reply to Joey Lee from comment #29)
> > (In reply to Joey Lee from comment #28)
> > > Reopen this issue against ALP.(In reply to Franck Bui from comment #27)
> > > > (In reply to Joey Lee from comment #25)
> > > > > The pesign-obs-integration be upgraded to 10.2+git20230612.4699910 on
> > > > > Base:System and openSUSE:Factory.
> > > > 
> > > > Can IBS also be fixed so ALP won't be hit by the same issue ?
> > > 
> > > Reopen this issue against ALP.
> > 
> > I have submit 10.2+git20230612.4699910 pesign-obs-integration from
> > openSUSE:Factory to SUSE:ALP:Workbench:1.0 on IBS.
> > 
> > submitreq is:
> > 
> > https://build.suse.de/request/show/301904
> 
> Re-submitreq to SUSE:ALP:Source:Standard:1.0 because SUSE:ALP:Workbench:1.0
> is frozen:
> 
> https://build.suse.de/request/show/302111

The submitreq be merged to SUSE:ALP:Source:Standard:1.0. 

Set issue to FIXED.