Bug 1164454

Summary: duplicated binaries between llvm9 and llvm7
Product: [openSUSE] openSUSE Distribution Reporter: Max Lin <mlin>
Component: OtherAssignee: Richard Biener <rguenther>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P2 - High CC: aaronpuchert, alarrosa, gnome-bugs, lubos.kocman, mlin, rguenther
Version: Leap 15.2   
Target Milestone: ---   
Hardware: Other   
OS: Other   
See Also: https://bugzilla.opensuse.org/show_bug.cgi?id=1145735
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Max Lin 2020-02-20 10:01:42 UTC
llvm9[1] is newly added to SLE15 SP2 and was arrived to Leap, we see duplicated binaries issue between llvm9 and llvm7, see below,

i586:
  python3-clang:
  - llvm7
  - llvm9
x86_64:
  libc++-devel:
  - llvm7
  - llvm9
  libc++1:
  - llvm7
  - llvm9
  libc++abi-devel:
  - llvm7
  - llvm9
  libc++abi1:
  - llvm7
  - llvm9
  python3-clang:
  - llvm7
  - llvm9

llvm7 in SLE15 SP2 hasn't build but inherited from update channel thus this isn't a problem there, however we need to find a solution for Leap.

Richard, seems it's not yet to have bugowner of llvm9 in SLE, I added you to CC'ed due to llvm9 in SLE15 SP2 was submitted by you.

[1] https://build.opensuse.org/request/show/774578
Comment 1 Richard Biener 2020-02-20 10:22:54 UTC
What are the actual duplicate binaries?  For SLE15 none fo the above cited
sub-packages should make it to the media, for Leap we probably want the
full llvm9 though.

libc++1, yes - doing it properly would need some similar mechanism as
we have for the GCC runtime (some %define as to which one becoming
the main one in the project config and suffixing the other).

What's the solution for Factory here?  Ah, we update the old package to
no longer build libcxx and python3-clang.

The spec file says

# install python bindings
# The python bindings use the unversioned libclang.so,
# so it doesn't make sense to have multiple versions of it
%if %{with pyclang}
install -d %{buildroot}%{python3_sitelib}/clang

Note applying the Factory solution to SLE would mean updating llvm7 in
SP2 and thus splitting it, requiring future maint updates to be done
twice, once for SP1 and once for SP2+ which doesn't sound appealing to me.

Can't you apply some magic to the "product config" to prefer the llvm9
variants for Leap?
Comment 2 Richard Biener 2020-02-20 10:24:33 UTC
(In reply to Richard Biener from comment #1) 
> Note applying the Factory solution to SLE would mean updating llvm7 in
> SP2 and thus splitting it, requiring future maint updates to be done
> twice, once for SP1 and once for SP2+ which doesn't sound appealing to me.

Also as customers should still see and use the llvm7 variants of those
packages since llvm9 only materializes as libLLVM9.so and libclang9.so
and nothing else.
Comment 3 Max Lin 2020-02-20 10:46:07 UTC
(In reply to Richard Biener from comment #1)
> What are the actual duplicate binaries?  For SLE15 none fo the above cited
> sub-packages should make it to the media, for Leap we probably want the
> full llvm9 though.

The actual duplicated binaries:
libc++-devel, libc++1, libc++abi-devel, libc++abi1 and python3-clang RPMs, llvm7 and llvm9 both had provide those RPMs. For Leap, if we had both full llvm9 and full llvm7, OBS scheduler will just picks one of duplicated RPM if any package build requires it.

> 
> libc++1, yes - doing it properly would need some similar mechanism as
> we have for the GCC runtime (some %define as to which one becoming
> the main one in the project config and suffixing the other).
> 
> What's the solution for Factory here?  Ah, we update the old package to
> no longer build libcxx and python3-clang.
> 
> The spec file says
> 
> # install python bindings
> # The python bindings use the unversioned libclang.so,
> # so it doesn't make sense to have multiple versions of it
> %if %{with pyclang}
> install -d %{buildroot}%{python3_sitelib}/clang
> 
> Note applying the Factory solution to SLE would mean updating llvm7 in
> SP2 and thus splitting it, requiring future maint updates to be done
> twice, once for SP1 and once for SP2+ which doesn't sound appealing to me.
> 
> Can't you apply some magic to the "product config" to prefer the llvm9
> variants for Leap?

for duplicated RPMs, prefer llvm9 variants has not fix this issue, Prefer: is used for RPM.
Comment 4 Richard Biener 2020-02-20 11:29:04 UTC
Ah, binaries == RPM packages.

The scheduler usually complains with "have choice for" but it may still have
the very old issue with RPMs with the same basename of being random there.

Note I cannot spend any time on this issue right now so let me suggest to
drop both the Mesa and llvm9 updates from Leap for now?  The different
build setup of Leap makes it quite difficult to find a solution that works
for both SLE and Leap.
Comment 5 Max Lin 2020-02-20 11:33:53 UTC
Mesa and llvm9 are still in Leap's staging project, I don't dare to merge them due to this issue, so I'll move them to ignore list until we have a solution.
Comment 6 Aaron Puchert 2020-02-20 23:48:31 UTC
(In reply to Richard Biener from comment #1)
> What's the solution for Factory here?  Ah, we update the old package to
> no longer build libcxx and python3-clang.
Correct, this is documented in README.packaging in the llvm metapackage. (Perhaps not the place one would expect it...)

> Note applying the Factory solution to SLE would mean updating llvm7 in
> SP2 and thus splitting it, requiring future maint updates to be done
> twice, once for SP1 and once for SP2+ which doesn't sound appealing to me.
Are the packages otherwise shared between SPs? Because Leap has different repos, as far as I know. And so far I didn't run into rebase conflicts because of turning off the libc++ builds for older packages.

By the way, we should also update the llvm metapackage (https://build.opensuse.org/package/show/openSUSE:Leap:15.2/llvm), so that llvm9 would be used by most packages. I don't think that defaulting to the old version is a good idea.

There was also a major version update to LLVM between Leap 15.0 and 15.1, specifically from 5 to 7. So I don't think there is anything new here. We'd just need to repeat what was done back then.

So I'd prefer the Factory solution for Leap 15.2: disable the libc++ & python-clang builds for llvm7, update the llvm metapackage to 9.0.1.
Comment 7 Richard Biener 2020-02-21 08:35:10 UTC
(In reply to Aaron Puchert from comment #6)
> (In reply to Richard Biener from comment #1)
> > What's the solution for Factory here?  Ah, we update the old package to
> > no longer build libcxx and python3-clang.
> Correct, this is documented in README.packaging in the llvm metapackage.
> (Perhaps not the place one would expect it...)
> 
> > Note applying the Factory solution to SLE would mean updating llvm7 in
> > SP2 and thus splitting it, requiring future maint updates to be done
> > twice, once for SP1 and once for SP2+ which doesn't sound appealing to me.
> Are the packages otherwise shared between SPs? Because Leap has different
> repos, as far as I know. And so far I didn't run into rebase conflicts
> because of turning off the libc++ builds for older packages.

Yes, in SLE the actual binary rpm packages are shared.

> By the way, we should also update the llvm metapackage
> (https://build.opensuse.org/package/show/openSUSE:Leap:15.2/llvm), so that
> llvm9 would be used by most packages. I don't think that defaulting to the
> old version is a good idea.
> 
> There was also a major version update to LLVM between Leap 15.0 and 15.1,
> specifically from 5 to 7. So I don't think there is anything new here. We'd
> just need to repeat what was done back then.

This was also done for SLE15 SP1 but there llvm5 was not adjusted (the
issue doesn't arise there since the setup is different - a service pack
is just a set of update rpms ontop of the GA product).  If we ever need
to update llvm5 we'd need to disable the pieces that would otherwise conflict.

> So I'd prefer the Factory solution for Leap 15.2: disable the libc++ &
> python-clang builds for llvm7, update the llvm metapackage to 9.0.1.

That works for me but then the origin should be the Factory packages
rather than the SLE ones.  Note that in SLE15 SP2 we have quite some
packages that use clang/llvm-devel and all of those (but Mesa!) still
use llvm7 - I'm not sure whether they build if we update the 'llvm'
package or which one would need adjustment to build-require llvm7/clang7
variants.  This and because the "update" for SP2 was late is the reason
I refrained from touching the 'llvm' meta-package and made sure we only
use llvm9 from Mesa (where it was requested as a feature) for SP2.

I also wonder if we want to drop llvm5/7 from Leap 15.2 if we introduce
llvm9.

But as said all this is quite some work and I currently have no spare
cycles for a proper solution in Leap.

We can of course try pulling llvm and llvm9 from Factory into a staging
and see what breaks...
Comment 8 Aaron Puchert 2020-02-21 11:57:26 UTC
(In reply to Richard Biener from comment #7)
> That works for me but then the origin should be the Factory packages
> rather than the SLE ones.  Note that in SLE15 SP2 we have quite some
> packages that use clang/llvm-devel and all of those (but Mesa!) still
> use llvm7 - I'm not sure whether they build if we update the 'llvm'
> package or which one would need adjustment to build-require llvm7/clang7
> variants.  This and because the "update" for SP2 was late is the reason
> I refrained from touching the 'llvm' meta-package and made sure we only
> use llvm9 from Mesa (where it was requested as a feature) for SP2.
Ok, I'm not sure what's in SLE that depends on LLVM, might have a look later today.

> I also wonder if we want to drop llvm5/7 from Leap 15.2 if we introduce
> llvm9.
I've already filed (https://build.opensuse.org/request/show/776163) for deletion of llvm5, because it's no longer required as dependency. We can't drop 7 though, because a handful of packages still want that. (ghc, beignet, python-llvmlite...)

> But as said all this is quite some work and I currently have no spare
> cycles for a proper solution in Leap.
Yeah, the different setup of SLE and Leap makes it a bit harder to find a solution that works for both.

> We can of course try pulling llvm and llvm9 from Factory into a staging
> and see what breaks...
llvm9 could still come from SLE, then we wouldn't violate the idea of having the base packages come from SLE. Only the metapackage would come from Factory.

To drop the duplicate llvm7 packages, we could just take the version from Factory as well. I think diverging from SLE in the light of this bug should be Ok. We've diverged anyway after my fix to bug 1138457. That request went in despite the origin change. (https://build.opensuse.org/request/show/712921)
Comment 9 Aaron Puchert 2020-02-21 12:09:57 UTC
I've filed submit requests for llvm [1] and llvm7 [2] from Factory to Leap 15.2, let's see where this goes.

[1] https://build.opensuse.org/request/show/777941
[2] https://build.opensuse.org/request/show/777943
Comment 10 Max Lin 2020-02-24 08:38:09 UTC
FYI I leave those SRs on backlog, for diverging from SLE we need agreement from release manager and package maintainer.
Comment 11 Aaron Puchert 2020-02-24 11:08:49 UTC
My idea was that we stage this and try if it works, as Richard Biener suggested in comment 7. If it doesn't work, there is no point in discussing it.
Comment 12 Max Lin 2020-02-27 07:25:27 UTC
(In reply to Aaron Puchert from comment #11)
> My idea was that we stage this and try if it works, as Richard Biener
> suggested in comment 7. If it doesn't work, there is no point in discussing
> it.

Ok, let's see how it works in Leap staging https://build.opensuse.org/project/show/openSUSE:Leap:15.2:Staging:E
Comment 13 Max Lin 2020-03-02 08:48:06 UTC
As Aaron has mentioned at https://build.opensuse.org/request/show/774578 , rust is build failed with llvm9, rust is also from SLE thus it's potentially rust should be build fail also in SLE...
Comment 14 Richard Biener 2020-03-02 09:12:16 UTC
(In reply to Max Lin from comment #13)
> As Aaron has mentioned at https://build.opensuse.org/request/show/774578 ,
> rust is build failed with llvm9, rust is also from SLE thus it's potentially
> rust should be build fail also in SLE...

SLE has 'llvm' still at version 7 so packages wanting to use llvm9 need to
explicitely build-require llvm9-devel and friends.  Only Mesa-drivers does that
at the moment.
Comment 15 Aaron Puchert 2020-03-02 22:38:03 UTC
(In reply to Richard Biener from comment #14)
> SLE has 'llvm' still at version 7 so packages wanting to use llvm9 need to
> explicitely build-require llvm9-devel and friends.  Only Mesa-drivers does
> that at the moment.
Correct, which is a bit of a problem though (or could be): linking with multiple versions of libLLVM.so is not supported, but Mesa consists of libraries, so if an application links with the default libLLVM.so.7 and has a graphical interface, it will also link with libLLVM.so.9 through Mesa at runtime. That doesn't work.

That's why I think Mesa should follow the system-wide default.

Rust on the other hand could still link with libLLVM.so.7, because it's a leaf executable that doesn't have anything to do with Mesa.

Side note: the libraries in Mesa-gallium link with libLLVM, and the library in Mesa-libOpenCL links additionally with libclang-cpp.
Comment 16 Max Lin 2020-03-02 23:08:55 UTC
(In reply to Richard Biener from comment #14)
> (In reply to Max Lin from comment #13)
> > As Aaron has mentioned at https://build.opensuse.org/request/show/774578 ,
> > rust is build failed with llvm9, rust is also from SLE thus it's potentially
> > rust should be build fail also in SLE...
> 
> SLE has 'llvm' still at version 7 so packages wanting to use llvm9 need to
> explicitely build-require llvm9-devel and friends.  Only Mesa-drivers does
> that
> at the moment.

ah right! I forgot we had llvm change in the same staging group, I should think twice before leave the comment...silly me.

yes, Mesa update has explicity build requires llvm9-devel therefore Mesa update can't accept to Leap until this issue has resolved.
Comment 17 Aaron Puchert 2020-03-04 00:36:49 UTC
My suggestion would be to change rust in SLE and Leap to require llvm7-devel instead of llvm-devel >= 6.0 and update the llvm metapackage to version 9.

Otherwise we're just going to run into issues with packages like kdevelop5 or libqt5-creator that use libLLVM and Mesa.
Comment 18 Richard Biener 2020-03-04 07:48:16 UTC
(In reply to Aaron Puchert from comment #17)
> My suggestion would be to change rust in SLE and Leap to require llvm7-devel
> instead of llvm-devel >= 6.0 and update the llvm metapackage to version 9.
> 
> Otherwise we're just going to run into issues with packages like kdevelop5
> or libqt5-creator that use libLLVM and Mesa.

I'm not against such change, it would be an update to SLE15-SP1 rust
(we don't want to split sources for this).  Given rust isn't happy with
llvm9 the >= 6.0 check is "bogus" anyways.  There's only a group of maintainers
though, CCing.

So if we "fix" rust then pulling 'llvm' from Factory into Leap works?  Can you
stage that (change rust in Leap) to try?
Comment 19 Max Lin 2020-03-05 09:47:39 UTC
(In reply to Richard Biener from comment #18)
> (In reply to Aaron Puchert from comment #17)
> > My suggestion would be to change rust in SLE and Leap to require llvm7-devel
> > instead of llvm-devel >= 6.0 and update the llvm metapackage to version 9.
> > 
> > Otherwise we're just going to run into issues with packages like kdevelop5
> > or libqt5-creator that use libLLVM and Mesa.
> 
> I'm not against such change, it would be an update to SLE15-SP1 rust
> (we don't want to split sources for this).  Given rust isn't happy with
> llvm9 the >= 6.0 check is "bogus" anyways.  There's only a group of
> maintainers
> though, CCing.
> 
> So if we "fix" rust then pulling 'llvm' from Factory into Leap works?  Can
> you
> stage that (change rust in Leap) to try?

FYI I've added rust to staging E[1] from Factory, the rust version is 1.40, there is no build error appeared in staging E at least.

[1] https://build.opensuse.org/project/show/openSUSE:Leap:15.2:Staging:E
Comment 20 Antonio Larrosa 2020-03-19 09:37:25 UTC
Since rust 1.40 builds fine with llvm9 in Staging:E and openQA tested the resulting Firefox works fine (https://openqa.opensuse.org/tests/1204998#step/firefox/1), I've submitted rust 1.40 to SLE-15-SP2 in https://build.suse.de/request/show/214229 .
Comment 21 Stefan Weiberg 2020-03-19 16:43:50 UTC
I would really like to avoid putting a big version update of rust (1.36 to 1.40) into SLE after we are just now releasing RC1. General rule is that during RC Phase we are only accepting P1 fixes and nasty P2 fixes into the product.

Could we instead use the sources we have in SLE right now and apply the spec changes to let it build with llvm9?
Comment 22 Aaron Puchert 2020-03-19 21:50:01 UTC
(In reply to Stefan Weiberg from comment #21)
> I would really like to avoid putting a big version update of rust (1.36 to
> 1.40) into SLE after we are just now releasing RC1. General rule is that
> during RC Phase we are only accepting P1 fixes and nasty P2 fixes into the
> product.
> 
> Could we instead use the sources we have in SLE right now and apply the spec
> changes to let it build with llvm9?

That's not so easy, the rust guys usually have to put in quite a bit of work to keep up with the API churn between major LLVM releases, see for example this PR for LLVM 9 support: https://github.com/rust-lang/rust/pull/62474. That's in rust 1.38 earliest, if I read GitHub correctly, so you'd have to port that back.

That's why I proposed to change rust to use llvm7-devel but otherwise update to LLVM 9 as default.

On the other hand rust developers love to live on the bleeding edge, so maybe they'd actually appreciate the update? I don't know.
Comment 24 Antonio Larrosa 2020-03-24 15:41:44 UTC
In https://build.suse.de/request/show/214351 I backported the patches from https://github.com/rust-lang/rust/pull/62474 (thanks Aaron for the link) to let rust 1.36 build with llvm9, so it should fix this issue when it gets to Leap.
Comment 25 Swamp Workflow Management 2020-04-01 13:19:19 UTC
SUSE-RU-2020:0839-1: An update that has one recommended fix can now be installed.

Category: recommended (moderate)
Bug References: 1164454
CVE References: 
Sources used:
SUSE Linux Enterprise Module for Open Buildservice Development Tools 15-SP1 (src):    rust-1.36.0-7.1
SUSE Linux Enterprise Module for Development Tools 15-SP1 (src):    rust-1.36.0-7.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 26 Max Lin 2020-05-29 15:32:07 UTC
Since llvm* change for this issue hasn't go further in SLE side, after talked to Lubos, we agreed to merge forked llvm/llvm7 in Leap in order to have llvm9 and Mesa to be acceptable(with additional libqt5-creator, etc.).
Comment 27 Richard Biener 2020-08-26 08:41:59 UTC
Thus fixed.