Bug 981083 - rpm ksyms fileattrs do not expand %name
rpm ksyms fileattrs do not expand %name
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Basesystem
Current
Other Other
: P5 - None : Normal (vote)
: ---
Assigned To: Michal Marek
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2016-05-23 10:05 UTC by Jiri Slaby
Modified: 2019-01-18 07:41 UTC (History)
8 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments
[PATCH] Fix passing package name and spec file to find-supplements.ksyms (2.29 KB, patch)
2016-09-26 15:51 UTC, Michal Marek
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jiri Slaby 2016-05-23 10:05:31 UTC
When the kernel is built in OBS, provides are being found. But this emits a lot of:
[  400s] sed: can't read /usr/src/packages/SOURCES/%name.spec: No such file or directory

The process tree is:
    \_ rpmbuild -ba --define _srcdefattr (-,root,root) --nosignature /home/abuild/rpmbuild/SOURCES/kernel-default.spec  
        \_ /bin/bash /home/abuild/rpmbuild/SOURCES/find-provides kernel-default
            \_ /bin/sh /usr/lib/rpm/find-provides kernel-default
                \_ /usr/lib/rpm/rpmdeps --provides
                    \_ /bin/bash /usr/lib/rpm/find-supplements.ksyms %name
                        \_ /bin/bash /usr/lib/rpm/find-supplements.ksyms %name                          |   \_ /bin/bash /usr/lib/rpm/find-supplements.ksyms %name
                        |   \_ sed -nre s,(.+:.+),modalias(kernel-default:\1),p  
                        |   \_ tee -a /tmp/tmp.6NZ7l2NDRz
                        \_ sort -u
                        \_ /bin/bash /usr/lib/rpm/find-supplements.ksyms %name


It can be seen, that this is executed:
/usr/lib/rpm/find-supplements.ksyms %name

I believe this is from /usr/lib/rpm/fileattrs/ksyms.attr:
%__ksyms_provides       %{_rpmconfigdir}/find-provides.ksyms %name
%__ksyms_requires       %{_rpmconfigdir}/find-requires.ksyms %name
%__ksyms_supplements    %{_rpmconfigdir}/find-supplements.ksyms %name
%__ksyms_path   (/lib/modules/.*\.ko(\.gz)?)|(/boot/vmlinu[xz].*)$


But %name is never expanded here.
Comment 1 Jiri Slaby 2016-05-23 10:36:17 UTC
Should find-provides do something like:
/usr/lib/rpm/rpmdeps --provides -D"name $1"
?
Comment 2 Jiri Slaby 2016-09-14 15:27:40 UTC
Michal has this on a todo list. Please close this if there is a dup somewhere or once it is fixed.
Comment 3 Michal Marek 2016-09-26 14:37:55 UTC
It puzzles me why we call /usr/lib/rpm/find-supplements.ksyms via rpmdeps --provides and rpmdeps --requires. There is a --supplements option to rpmdeps. This is however not the cause of the failed expansion.
Comment 4 Michal Marek 2016-09-26 14:46:18 UTC
Actually, rpm 4.12.0 does not have it. I was looking at the git master branch.
Comment 5 Michal Marek 2016-09-26 14:55:35 UTC
The behavior is still weird. Looks like any time rpmdeps is invoked, we also invoke all tke find-*.ksyms helpers.

 rpmdeps(/usr/lib/rpm/rpmdeps) --provides
  +-- find-provides.ksyms(/usr/lib/rpm/find-provides.ksyms) %name
  |    +-- (anon)
  |    `-- sort -u
  +-- find-requires.ksyms(/usr/lib/rpm/find-requires.ksyms) %name
  |    +-- (anon)
  |    `-- sort -u
  `-- find-supplements.ksyms(/usr/lib/rpm/find-supplements.ks...) %name
       +-- (anon)
Comment 6 Michal Marek 2016-09-26 15:51:38 UTC
Created attachment 694289 [details]
[PATCH] Fix passing package name and spec file to  find-supplements.ksyms

This patch to rpm fixes the error for me. However, there is still a problem that with _use_internal_dependency_generator 0, the output of find-supplements.ksyms seems ignored and no supplements are generated. This is not a problem for the kernel packages, but it can be seen by building e.g. openSUSE:Factory/bbswitch with %define _use_internal_dependency_generator 0.
Comment 7 Michal Marek 2016-09-26 15:52:12 UTC
Michael, can you check the patch?
Comment 8 Michael Schröder 2016-09-27 09:14:03 UTC
Turning of the internal dependency generator is not supported, all packages that do that are broken.

Looking at the rpm code it seems that find-provides is only called if the internal dependency generator is *off*. What are you guys doing? I do not want to support this!

(As for the patch, what's with those "if" statements when setting the env?)
Comment 9 Michael Schröder 2016-09-27 09:18:28 UTC
Note that upstream rpm will print a "Deprecated external dependency generator is used!" warning. The external generator code is going to be removed...
Comment 10 Michal Marek 2016-09-30 13:30:47 UTC
How do you suggest to implement per-subpackage custom generated dependencies? Not everything can be keyed off individual files...
Comment 11 Michael Schröder 2016-09-30 13:33:12 UTC
Why would that be an issue? You still get passed the %name to the script with the internal generator, right?
Comment 12 Michal Marek 2016-09-30 13:48:19 UTC
To give a more concrete example: kernel-default.spec in SP2 produces a handful of these "KMPs":

cluster-md-kmp-default
cluster-network-kmp-default
dlm-kmp-default
gfs2-kmp-default
ocfs2-kmp-default

Now all of them depend on the exact version of the main kernel package (we are not using the ksym() dependencies as we are not sharing the KMPs among different kernel versions) and there are some dependencies among the packages themselves (gfs2 and ocfs2 require dlm I think). In order to avoid hard-coding the dependencies in the spec file, the find-requires script in kernel-source checks 1) if the given set of *.ko files is one of those KMPs, it generates a dependency kernel-default = %version-%release 2) if any of the *.ko files depend on *.ko files from another KMP, it generates a dependency like dlm-kmp-default = %version-%release. All this requires knowing to which subpackage do the files on input belong and I'm not sure if the /usr/lib/rpm/fileattrs based dependency generator has such information. Of course, we could generate the ksym() dependencies and *somehow* inject at least the dependency on kernel-default. But it would allow the solver to make choices like "I'll give you this version of kernel-default and ocfs2-kmp-default, and since the latter has some funny dependencies, I'll also give you this other version of dlm-kmp-default and the kernel it depends on". I *hope* the solver is smart enough not to do this, but expressing the dependencies explicitly is IMO a safer choice.
Comment 13 Michal Marek 2016-09-30 13:49:17 UTC
(In reply to Michael Schröder from comment #11)
> Why would that be an issue? You still get passed the %name to the script
> with the internal generator, right?

I need the name of the subpackage. Can I get it with the internal generator?
Comment 14 Andreas Färber 2017-04-19 21:47:29 UTC
This issue is still around a year later...
Comment 15 Michal Marek 2017-06-09 14:05:04 UTC
My current plan is to give up on the automatic dependencies between dlm-kmp-default and ocfs-kmp-default & co and just hardcode them somewhere. Then we should be able to use the internal dependency generator. I will also remove the undocumented support for %supplements_kernel_module, which is the reason for the sed error.
Comment 16 Michal Marek 2017-06-14 13:53:20 UTC
I made some cleanup of the kernel dependency helpers here

  https://build.opensuse.org/project/show/home:michal-m:branches:Base:System

In particular, the support for %supplements_kernel_module is dropped, so we do not need to open the spec file from the helper. Also, the helpers are invoked based on paths below /lib/modules, so we do not need %name at all and the build of the kernel does not spawn the requires and supplements helper 3800 times. Michael, what do you think?

For reference, this is the SLE12 version of the changes:

  https://build.opensuse.org/project/show/home:michal-m:branches:SUSE:SLE-12:Update

I'm not proposing any change in SLE12, but the requires/provides helper will need to be ported to SLE15.
Comment 17 Michal Marek 2017-08-30 13:24:25 UTC
I rebased the changes and submitted to Base:System (SR#519504). I also re-enabled the ksym() dependencies and submitted to SLE15 (SR#139447 in the IBS).
Comment 18 Michal Marek 2017-08-30 14:31:08 UTC
Now Frederic wants me to make the ksym() provides conditional on %is_opensuse so that the sources can be shared.
Comment 19 Michael Schröder 2017-08-31 13:25:18 UTC
Hey, good idea! We should have done this a long time ago.
Comment 20 Michal Marek 2017-09-11 12:21:21 UTC
This is now done: https://build.opensuse.org/request/show/523080.
Comment 23 Swamp Workflow Management 2018-11-06 15:45:40 UTC
This is an autogenerated message for OBS integration:
This bug (981083) was mentioned in
https://build.opensuse.org/request/show/646686 42.3 / kernel-source
Comment 25 Swamp Workflow Management 2018-11-09 20:46:08 UTC
SUSE-SU-2018:3688-1: An update that solves 6 vulnerabilities and has 100 fixes is now available.

Category: security (important)
Bug References: 1011920,1012382,1012422,1020645,1031392,1035053,1042422,1043591,1048129,1050431,1050549,1053043,1054239,1057199,1065600,1065726,1067906,1073579,1076393,1078788,1079524,1082519,1083215,1083527,1084760,1089343,1091158,1093118,1094244,1094825,1095805,1096052,1098050,1098996,1099597,1101555,1103308,1103405,1104124,1105025,1105428,1105795,1105931,1106105,1106110,1106240,1106293,1106359,1106434,1106594,1106913,1106929,1107060,1107299,1107318,1107535,1107829,1107870,1108315,1108377,1108498,1109158,1109333,1109772,1109784,1109806,1109818,1109907,1109919,1109923,1110006,1110363,1110468,1110600,1110601,1110602,1110603,1110604,1110605,1110606,1110611,1110612,1110613,1110614,1110615,1110616,1110618,1110619,1111363,1111516,1111870,1112007,1112262,1112263,1112894,1112902,1112903,1112905,1113667,1113751,1113769,1114178,1114229,1114648,981083,997172
CVE References: CVE-2018-14633,CVE-2018-18281,CVE-2018-18386,CVE-2018-18690,CVE-2018-18710,CVE-2018-9516
Sources used:
SUSE Linux Enterprise Live Patching 12-SP3 (src):    kgraft-patch-SLE12-SP3_Update_21-1-4.5.2
Comment 26 Swamp Workflow Management 2018-11-12 22:06:17 UTC
This is an autogenerated message for OBS integration:
This bug (981083) was mentioned in
https://build.opensuse.org/request/show/648620 42.3 / kernel-source
Comment 27 Swamp Workflow Management 2018-11-20 20:20:36 UTC
openSUSE-SU-2018:3817-1: An update that solves 5 vulnerabilities and has 66 fixes is now available.

Category: security (important)
Bug References: 1011920,1012382,1012422,1020645,1031392,1035053,1042422,1043591,1048129,1050431,1053043,1054239,1057199,1062303,1065600,1065726,1067906,1073579,1076393,1078788,1079524,1083215,1083527,1084760,1091158,1093118,1094825,1095805,1098050,1098996,1101555,1104124,1105025,1105931,1106110,1106359,1106594,1106913,1106929,1107060,1107299,1107535,1107870,1108377,1108498,1109158,1109772,1109784,1109818,1109907,1109919,1109923,1110006,1111516,1111870,1112007,1112262,1112263,1112894,1112902,1112903,1112905,1113667,1113751,1113769,1114178,1114229,1114648,1115587,981083,997172
CVE References: CVE-2018-18281,CVE-2018-18386,CVE-2018-18690,CVE-2018-18710,CVE-2018-9516
Sources used:
openSUSE Leap 42.3 (src):    kernel-debug-4.4.162-78.1, kernel-default-4.4.162-78.1, kernel-docs-4.4.162-78.1, kernel-obs-build-4.4.162-78.1, kernel-obs-qa-4.4.162-78.1, kernel-source-4.4.162-78.1, kernel-syms-4.4.162-78.1, kernel-vanilla-4.4.162-78.1
Comment 30 Swamp Workflow Management 2019-01-16 08:07:01 UTC
SUSE-SU-2019:0095-1: An update that solves 13 vulnerabilities and has 140 fixes is now available.

Category: security (important)
Bug References: 1011920,1012382,1012422,1020645,1031392,1035053,1042422,1043591,1044189,1048129,1050431,1050549,1053043,1054239,1057199,1062303,1063026,1065600,1065726,1066223,1067906,1073579,1076393,1078788,1079524,1082519,1082863,1082979,1083215,1083527,1084427,1084536,1084760,1087209,1088087,1089343,1090535,1091158,1093118,1094244,1094555,1094562,1094825,1095344,1095753,1095805,1096052,1096547,1098050,1098996,1099597,1099810,1101555,1102495,1102715,1102870,1102875,1102877,1102879,1102882,1102896,1103156,1103269,1103308,1103405,1104124,1105025,1105428,1105795,1105931,1106095,1106105,1106110,1106240,1106293,1106359,1106434,1106512,1106594,1106913,1106929,1106934,1107060,1107299,1107318,1107535,1107829,1107870,1107924,1108096,1108170,1108240,1108281,1108315,1108377,1108399,1108498,1108803,1108823,1109038,1109158,1109333,1109336,1109337,1109441,1109772,1109784,1109806,1109818,1109907,1109919,1109923,1110006,1110297,1110337,1110363,1110468,1110600,1110601,1110602,1110603,1110604,1110605,1110606,1110611,1110612,1110613,1110614,1110615,1110616,1110618,1110619,1110930,1111363,1111516,1111870,1112007,1112262,1112263,1112894,1112902,1112903,1112905,1113667,1113751,1113766,1113769,1114178,1114229,1114648,1115593,981083,997172
CVE References: CVE-2018-14613,CVE-2018-14617,CVE-2018-14633,CVE-2018-16276,CVE-2018-16597,CVE-2018-17182,CVE-2018-18281,CVE-2018-18386,CVE-2018-18690,CVE-2018-18710,CVE-2018-7480,CVE-2018-7757,CVE-2018-9516
Sources used:
SUSE Linux Enterprise Server 12-SP3 (src):    kernel-azure-4.4.162-4.19.2, kernel-source-azure-4.4.162-4.19.1, kernel-syms-azure-4.4.162-4.19.1