Bug 1214303

Summary: In zypper dist-upgrade, 79 "keep obsolete" due to perl-5.38.0
Product: [openSUSE] openSUSE Tumbleweed Reporter: James Carter <jimc>
Component: libzyppAssignee: Michael Schröder <mls>
Status: NEW --- QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: jimc, lars.vogdt
Version: Current   
Target Milestone: ---   
Hardware: x86-64   
OS: openSUSE Tumbleweed   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: Transcript of "zypper dist-upgrade" with enough "keep obsolete" to get to a solution.

Description James Carter 2023-08-15 22:30:10 UTC
Created attachment 868819 [details]
Transcript of "zypper dist-upgrade" with enough "keep obsolete" to get to a solution.

Installed:  openSUSE-release-20230807-2436.1.x86_64 (Tumbleweed)
    perl-5.36.1-1.2.x86_64 and friends
Upgrade to: openSUSE-release-20230814-2450.1.x86_64
    perl-5.38.0-1.3.x86_64 and friends
Repo: Tumbleweed Main Repository (OSS)

This is a hybrid of a bug report, a feature request, and a request for
suggestions how to work around problems to make a dist-upgrade happen. 
Sorry if I've misjudged how you want this reported.  

The symptom is, when I do the dist-upgrade, it wants to upgrade to
perl-5.38.0, but I have a lot of less common Perl modules (from SuSE),
and keystone packages like gnumeric, which require various perl-5.36.x
--provides, which cannot be provided if perl-5.38.0 replaces
perl-5.36.1.  These modules do not (yet) have build updates in which
their --requires were updated to perl-5.38.0 and their test suites were
run (successfully).  

This kind of thing happens a lot, and I normally choose "keep obsolete
package" and wait for the dependent package's --requires to be updated,
which you guys usually take care of pretty promptly.  However in big
upgrades like for Perl or PHP, the manual intervention on my end gets
horrendous (and I'm sure your workload is increased too) and I have to
pause updates for a while, which is not exactly a recommended response. 
That's why I'm reporting the problem as a bug.  

I've attached a transcript of picking "keep obsolete" 79 times, enough
to get to an upgrade solution.  (But I didn't actually let it upgrade,
to avoid having to remember that one machine is different from all the
others.)  The package "keystone-xena" is a meta-package that --requires
all packages that I want on the machine that "rpm -e --test pkg.rpm"
would not get an error on.  And it's locked.  Before I put in these
meta-packages, "zypper dist-upgrade" had a nasty habit of proposing to
remove 95% of the packages on the machine including the kernel, if there
were a dependency problem.  I was afraid of hitting the wrong response,
when doing this under manual supervision.

Here are two feature requests:

* In the spec files, version requirements are usually pointlike, e.g.
for gnumeric, perl = 5.36.1; perl(:MODULE_COMPAT_5.36.1).  Today's
problems could have been avoided if a range had been given, e.g.
perl >= 5.18.0 (earliest known backward compatible version), but I don't
know how that could be expressed for the MODULE_COMPAT requirement.  Also
you would have to conscientiously run the test suite with new dependency
versions, just as you need to do now with the pointlike dependencies. 

* In zypper it would really help if you had a configuration setting
giving the solver permission to automatically pick a solution containing
only "keep obsolete" actions.  Or during the solving process you could
make an individual "keep obsolete" an acceptable second choice of
strategy.  I've seen this feature requested several times but nobody
seems to know how it might be turned on.
Comment 1 Michael Andres 2023-08-24 16:18:00 UTC
@James, in case you finally performed the dup, there should be a solver-testcase at /var/log/updateTestcase-YYYY-MM-DD-hh-mm-ss (date and time of the dup).

If you don't mind, please pack the testcase directory attach it to the bugreport. It would help us to replay the conflicts and to test potential solutions.


  ##
  cd /tmp
  tar cvjf zypper-testcase.bz2 /var/log/updateTestcase-YYYY-MM-DD-hh-mm-ss

In case the testcase exceeds the 10MB limit for bugzilla uploads, you can use 'split' to divide the file into smaller pieces:

  cd /tmp
  split -b 10M --additional-suffix=-zypper-testcase.bz2 zypper-testcase.bz2

The created pieces will be named:

  xaa-zypper-testcase.bz2
  xab-zypper-testcase.bz2
  xac-...
Comment 2 James Carter 2023-08-25 22:26:02 UTC
I found the culprits: 
perl-Astro-Sunrise-0.99-1.26.noarch
    URL: https://metacpan.org/release/Astro-Sunrise
    Vendor: obs://build.opensuse.org/devel:languages:perl
    Build date: 2022-07-28
perl-Net-Wake-0.02-7.52.noarch
    URL: http://search.cpan.org/dist/Net-Wake/
    Vendor: obs://build.opensuse.org/devel:languages:perl
    Build date: 2022-06-23
These are found by the searcher https://software.opensuse.org/search
if you set it for "All Distributions", but not on "openSUSE Tumbleweed",
likely a clue why they didn't get updated to --require perl-5.38.0 .
They seem to work just fine if you "break" them by ignoring the 
dependency on perl-5.36.0 .  Then the dist-upgrade runs with no problems. 

I snooped around https://build.opensuse.org/project/show/devel:languages:perl
and spotted the "report bug" link.  Unfortunately my report was
rejected, "Bugzilla - Match Failed".  It seems that prefilled hidden 
fields for the assignee (stephan@kulow.org) and CC (lrupp@suse.com)
"did not match anything".  Do you suppose you could pass to the relevant
people a request to update the spec files of these two packages to
--require perl(:MODULE_COMPAT_5.38.0) rather than 
perl(:MODULE_COMPAT_5.36.0) since after the upgrade no package --provides
the latter?  (Like a reference to this bug report.)  

I assume you no longer need the test case, since "that's not a bug (in 
Zypper), that's a feature!"

Thank you for your work maintaining the SuSE distro.
Comment 3 Michael Andres 2023-08-28 13:52:49 UTC
@Michael, Lars? 
Please check the two packages mentioned in the previous comment.
(and the 'lrupp@suse.com' in obs)
Comment 4 Michael Schröder 2024-04-29 14:51:23 UTC
Perl does not provide API compatibility with the minor version, i.e. you can't run 5.36 binary modules with 5.38 perl.

The API is compatible with patch versions, that's why there are the perl(:MODULE_COMPAT_XXX) provides. If a new patch version is released, a new provides is added so that all the old patch versions are still provided.

The version requires is a result of the search path built into perl. For example, perl -V will show the following include path:

    /usr/lib/perl5/site_perl/5.38.2/x86_64-linux-thread-multi
    /usr/lib/perl5/site_perl/5.38.2
    /usr/lib/perl5/vendor_perl/5.38.2/x86_64-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.38.2
    /usr/lib/perl5/5.38.2/x86_64-linux-thread-multi
    /usr/lib/perl5/5.38.2
    /usr/lib/perl5/vendor_perl/5.38.0/x86_64-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.38.0
    /usr/lib/perl5/site_perl

As you can see, perl looks both in the 5.38.2 and 5.38.0 directories for modules. A module compiled for an old version is simply not in the search path and thus cannot be found. So the requires in the package is needed.