Bug 1219345

Summary: zypper dup with ZYPP_SINGLE_RPMTRANS=1 fails with kernel-default-6.7.1-2.1
Product: [openSUSE] openSUSE Tumbleweed Reporter: Martin Wilck <martin.wilck>
Component: libzyppAssignee: E-mail List <zypp-maintainers>
Status: RESOLVED UPSTREAM QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: db, fvogt, martin.wilck, roger.whittaker
Version: CurrentFlags: ma: needinfo?
Target Milestone: ---   
Hardware: Other   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Martin Wilck 2024-01-30 11:11:12 UTC
Current Tumbleweed (20240126). zypper dup wants to "reinstall" kernel-default-6.7.1-2.1

I am told this is because of the transition of the kernel switch to _multibuild (bsc#1218184), because the OBS source package of the kernel binary package is now kernel-source and not kernel-default as it used to be; the package has been rebuilt and some checksums don't match, while the version/release is the same as before.

# zypper dup
[TechPreview] $ZYPP_SINGLE_RPMTRANS=1 : New rpm install backend is enabled
              If you find any bugs or issues please let us know:
              https://bugzilla.opensuse.org/
              Component: libzypp (or zypper)
              And please attach the /var/log/zypper.log to the bug report.
Computing distribution upgrade...

The following 2 packages are going to be upgraded:
  openSUSE-release openSUSE-release-appliance-custom

The following product is going to be upgraded:
  openSUSE Tumbleweed  20240126-0 -> 20240128-0

The following package is going to be reinstalled:
  kernel-default-6.7.1-2.1

The following package requires a system reboot:
  kernel-default-6.7.1-2.1

2 packages to upgrade, 1 to reinstall.


With ZYPP_SINGLE_RPMTRANS=1, this reinstallation fails with file conflicts for essentially every single file. 

>    file /usr/lib/modules/6.7.1-2-default/kernel/sound/usb/line6/snd-usb-variax.ko.zst from install of kernel-default-6.7.1-2.1.x86_64 conflicts with file from package kernel-default-6.7.1-2.1.x86_64
>    file /usr/lib/modules/6.7.1-2-default/kernel/sound/usb/misc/snd-ua101.ko.zst from install of kernel-default-6.7.1-2.1.x86_64 conflicts with file from package kernel-default-6.7.1-2.1.x86_64
>    file /usr/lib/modules/6.7.1-2-default/kernel/sound/usb/snd-usb-audio.ko.zst from install of kernel-default-6.7.1-2.1.x86_64 conflicts with file from package kernel-default-6.7.1-2.1.x86_64
>    file /usr/lib/modules/6.7.1-2-default/kernel/sound/usb/snd-usbmidi-lib.ko.zst from install of kernel-default-6.7.1-2.1.x86_64 conflicts with file from package kernel-default-6.7.1-2.1.x86_64
>    file /usr/lib/modules/6.7.1-2-default/kernel/sound/usb/usx2y/snd-usb-us122l.ko.zst from install of kernel-default-6.7.1-2.1.x86_64 conflicts with file from package kernel-default-6.7.1-2.1.x86_64
Comment 1 Michael Andres 2024-01-30 11:53:42 UTC
(In reply to Martin Wilck from comment #0)
> the package has been rebuilt and some checksums don't match, while the
> version/release is the same as before.

That's the issue. It's no longer the same package with the same content but sill uses the same version-release. OBS should not do this. If the content changed the build count should be increased.

ZYPP_SINGLE_RPMTRANS most probably gives you the same file conflicts as a re-install with plain rpm does. That's the purpose of ZYPP_SINGLE_RPMTRANS.

IMO it's not a bug in zypp, but in OBS (unless rpm on the commandline would not complain)
Comment 2 Fabian Vogt 2024-01-30 12:17:47 UTC
But zypper realizes that it's a *re*installation, i.e. the old package is removed and the new one installed:

> The following package is going to be reinstalled:
>   kernel-default-6.7.1-2.1
Comment 3 Michael Andres 2024-01-30 12:25:09 UTC
No, a re-install is not 'remove and install'. 
It's an `update` to the same version (rpm -U --replacepkgs).
Comment 4 Fabian Vogt 2024-01-30 12:45:19 UTC
(In reply to Michael Andres from comment #3)
> No, a re-install is not 'remove and install'. 
> It's an `update` to the same version (rpm -U --replacepkgs).

Well, yes, I'm already a step ahead there. Main point is that after the update, the package is replaced, i.e. only one of either the old or new packages will be installed. So file conflicts can't happen. Otherwise no package update could ever be installed.

I guess there is some conflict with the multiversion functionality.
Comment 5 Martin Wilck 2024-01-30 14:02:58 UTC
(In reply to Michael Andres from comment #3)
> No, a re-install is not 'remove and install'. 
> It's an `update` to the same version (rpm -U --replacepkgs).

Not confirmed. I just ran 

rpm -U --replacepkgs /var/cache/zypp/packages/download.opensuse.org-oss/x86_64/kernel-default-6.7.1-2.1.x86_64.rpm

and I got no conflicts.
Comment 6 Michael Andres 2024-01-30 14:03:35 UTC
(In reply to Fabian Vogt from comment #4)
> I guess there is some conflict with the multiversion functionality.

You're probably right. With multiversion on it's `rpm -i --replacepkg`, so rpm does not obsoelete the old version(s).

It would all be no issue if OBS would increment the build count.
Comment 7 Michael Andres 2024-01-30 14:04:44 UTC
(In reply to Martin Wilck from comment #5)
> Not confirmed. I just ran 
> 
> rpm -U --replacepkgs

Maybe my fault, if multiversion is on: `rpm -i --replacepkg`
Comment 8 Martin Wilck 2024-01-30 14:07:39 UTC
Unfortunately the rpm -U destroyed my test bed ;-)
I'll have to roll back to test this.

It makes sense that this is caused by multiversion + the messed-up versioning of the kernel package. The missing piece (for me) is why it only happens with ZYPP_SINGLE_RPMTRANS=1.
Comment 9 Michael Andres 2024-01-30 14:19:47 UTC
(In reply to Martin Wilck from comment #8)
> happens with ZYPP_SINGLE_RPMTRANS=1.

Because with ZYPP_SINGLE_RPMTRANS librpm drives the whole transaction.

In the traditional installer we use a single 'rpm --shut-up-and-do-what-we-tell-;)' for every package. The file conflict check is done by libsolv because rpm itself never sees the whole transaction. Might be libsolv is more relaxed in this case. (@mls might know)
Comment 10 Martin Wilck 2024-01-30 14:20:40 UTC
(In reply to Michael Andres from comment #7)

> Maybe my fault, if multiversion is on: `rpm -i --replacepkg`

Indeed, this fails with the same messages as "zypper dup" with ZYPP_SINGLE_RPMTRANS=1.
Comment 11 Martin Wilck 2024-01-30 14:25:56 UTC
So my takeway here is that it's really unfortunate coincidence of multiversion, the logic of librpm, and the versioning of this specific kernel package in factory.

This constellation is unlikely to happen again any time soon. Thus I suppose it isn't worth putting in additional resources. Not many people seem to have been affected so far, and as soon as TW releases a new kernel, the issue will be gone.

We should keep an eye on the behavior of OBS though; I believe that version/release numbers should always increase in a distribution, and that our QA/release tools should catch this situation before it occurs.

As far as I'm concerned, the bug can be closed.
Comment 12 Michael Andres 2024-01-30 14:36:33 UTC
(In reply to Martin Wilck from comment #11)
> version/release numbers should always increase in a distribution, and that
> our QA/release tools should catch this situation before it occurs.

 +1

>As far as I'm concerned, the bug can be closed.
Comment 13 hui 2024-01-31 06:35:04 UTC
*** Bug 1219361 has been marked as a duplicate of this bug. ***