Bug 1218649 - [Build 129.7] SUSEConnect is broken: SLES to SLES for SAP Horizontal Migration fails on 15-SP5 QU
Summary: [Build 129.7] SUSEConnect is broken: SLES to SLES for SAP Horizontal Migratio...
Status: VERIFIED FIXED
Alias: None
Product: PUBLIC SUSE Linux Enterprise Server 15 SP5
Classification: openSUSE
Component: Registration (show other bugs)
Version: unspecified
Hardware: Other Other
: P1 - Urgent : Critical
Target Milestone: ---
Assignee: SCC Bugs
QA Contact: Hernan Schmidt
URL: https://openqa.suse.de/tests/13215340...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-01-09 13:41 UTC by Alvaro Carvajal
Modified: 2024-03-18 00:47 UTC (History)
13 users (show)

See Also:
Found By: openQA
Services Priority:
Business Priority:
Blocker: Yes
Marketing QA Status: ---
IT Deployment: ---


Attachments
y2logs (5.07 MB, application/x-xz)
2024-01-09 13:41 UTC, Alvaro Carvajal
Details
Screenshot after migration (64.00 KB, image/png)
2024-01-17 16:25 UTC, Alvaro Carvajal
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alvaro Carvajal 2024-01-09 13:41:42 UTC
Created attachment 871715 [details]
y2logs

* Platform and arch: x86_64 & ppc64le (on KVM)

* OS Version: SLES 15-SP5 QU Build 129.7
  - Installed via: ISO
  - Using procedure: manual installation in openQA

* LOGS
  - y2logs attached

* Results
  - Expected: SLES can be migrated to SLES for SAP using the migrate-sles-to-sles4sap package
  - Real: SUSEConnect fails to detect base product and migration fails

* Reproducible?: yes

* Way to reproduce it:
  - Boot a VM from the qcow2 image located in https://openqa.suse.de/tests/13215340/asset/hdd/SLE-15-SP5-x86_64-Build129.7@64bit-gnome-qr-sap.qcow2
  - This will boot a SLES system installed from SLES 15-SP5 QU Build 129.7
  - Register system to SCC
  - Log into the system and install the migrate-sles-to-sles4sap package
  - Run /usr/sbin/Migrate_SLES_to_SLES-for-SAP.sh
  - Script will fail with the messages:

###
Package 'sles-release-POOL' not found.

Unregistering the System...

Please be aware that the next step will take a few minutes.

As soon as it was successful you will receive a listing

of the subscribed channels.


Registering the default products and modules for

SUSE Linux Enterprise Server for SAP Applications 15 SP5...
SUSEConnect error: Unable to detect base product

The registration was NOT successful.
###

* openQA
  * Link to failed test: https://openqa.suse.de/tests/13215340#step/migrate_sles_to_sles4sap/35
  * Link to last successful run (build 123.1): https://openqa.suse.de/tests/12856760

* Description: Horizontal Migration (SLES to SLES for SAP) was working on the 15-SP5 QU product up until build 123.1. Starting on build 125.6 last month, this started to fail, with the migration script reporting that it was no able to detect the base product, and also reporting that the package sles-release-POOL could not be found. It seems like some external asset is missing or has been renamed.

## Reproducible

Fails since (at least) Build [125.6](https://openqa.suse.de/tests/12967999)

## Expected result

Last good: [123.1](https://openqa.suse.de/tests/12856760) (or more recent)

## Further details

Always latest result in this scenario: [latest](https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Online-QR-SAP&machine=64bit&test=sles4sap_horizontal_migration_saptune&version=15-SP5)
Comment 2 Marcus Meissner 2024-01-10 13:44:35 UTC
reassign to migrate-sles-to-sles4sap bugowner.
Comment 3 Marcus Meissner 2024-01-10 14:00:04 UTC
migrate-sles-to-sles4sap calls into SUSEConnect, aka suseconnect-ng.

there was a release between Nov 21 and Dec 5 which could have triggered this:

https://lists.suse.com/pipermail/sle-updates/2023-November/032957.html
Comment 4 Marcus Meissner 2024-01-10 14:11:27 UTC
might be this change removing the side effect of installation the SAP-release package:

commit 17affd73a90bb05abee0715642a4d7faf9cdc8cc
Author: Natnael Getahun <natnael.getahun@suse.com>
Date:   Thu Nov 23 15:00:39 2023 +0100

    Skip installing release package
Comment 5 Marcus Meissner 2024-01-10 14:11:57 UTC
blocks current Quarterly Update release.

also likely blocks this during customer scenarios since the release of suseconnect-ng
Comment 6 Marcus Meissner 2024-01-10 14:54:23 UTC
(as far as I read the migrate script it uses SUSEConnect to install the new release package.)
Comment 13 Volkan OZTUZUN 2024-01-15 15:11:28 UTC
Hello @Thomas Schmidt this is a blocker for QU2 that was supposed to have the release candidate tomorrow, January 16th. So we urgently need a target resolution date in order to understand how to adapt our plan and possibly keep the final announcement date on January 26th.
Please let us know.
Comment 14 Marcus Meissner 2024-01-15 15:16:57 UTC
To be clear, this must have regressed only in the last months, even with the suseconnect-ng already.

In 15-SP5 QU1 3-4 months ago this was still working.
Comment 15 Thomas Schmidt 2024-01-16 11:06:03 UTC
Could you please test with the latest suseconnect-ng 1.5.0 build from https://build.opensuse.org/package/show/systemsmanagement:SCC/suseconnect-ng ?
It should have this line in the changelog: 
'Allow horizontal migrations when showing EULAs (bsc#1218649 and bsc#1217961)'.
Comment 16 Alvaro Carvajal 2024-01-16 14:11:15 UTC
(In reply to Thomas Schmidt from comment #15)
> Could you please test with the latest suseconnect-ng 1.5.0 build from
> https://build.opensuse.org/package/show/systemsmanagement:SCC/suseconnect-ng
> ?
> It should have this line in the changelog: 
> 'Allow horizontal migrations when showing EULAs (bsc#1218649 and
> bsc#1217961)'.

I tried with suseconnect-ng-1.5.0~git3.c67e9c5-150500.74.1.x86_64 from https://download.opensuse.org/repositories/systemsmanagement:/SCC/SLE_15_SP5/.

I installed first suseconnect-ng-1.5.0~git3.c67e9c5-150500.74.1.x86_64, and then installed migrate-sles-to-sles4sap package, however migration is still failing.

Source package's changelog contains the change:

susetest:~ # rpm -q --changelog Downloads/suseconnect-ng-1.5.0_git3.c67e9c5-150500.74.1.src.rpm|grep -i horiz
warning: Downloads/suseconnect-ng-1.5.0_git3.c67e9c5-150500.74.1.src.rpm: Header V3 RSA/SHA256 Signature, key ID e6e5a213: NOKEY
  * Allow horizontal migrations when showing EULAs (bsc#1218649 and bsc#1217961)
Comment 17 Andreas Jaeger 2024-01-16 15:03:42 UTC
At what step is it failing now? Same as last time?
Comment 18 Alvaro Carvajal 2024-01-16 18:06:44 UTC
(In reply to Andreas Jaeger from comment #17)
> At what step is it failing now? Same as last time?

Same as last time. Error messages and retval are the same.
Comment 20 Thomas Schmidt 2024-01-17 09:51:20 UTC
We added another fix to handle that case. Please test with the package from https://download.opensuse.org/repositories/systemsmanagement:/SCC/ (it has ~git0.b838c19 as version).

I successfully verified it with: 

$ docker run registry.suse.com/suse/sle15:15.3
> zypper rm -y container-suseconnect
> zypper in -y https://download.opensuse.org/repositories/systemsmanagement:/SCC/SLE_15_SP3/x86_64/suseconnect-ng-1.5.0~git0.b838c19-150300.75.1.x86_64.rpm
> zypper in -y http://updates.suse.de/SUSE/Updates/SLE-Module-Server-Applications/15-SP3/x86_64/update/noarch/migrate-sles-to-sles4sap-15.1.2-3.9.1.noarch.rpm
> /usr/sbin/Migrate_SLES_to_SLES-for-SAP.sh
Comment 21 Andreas Jaeger 2024-01-17 12:58:17 UTC
Peter, Alvaro, could you test with the new package, please?
Comment 22 Peter Varkoly 2024-01-17 13:47:52 UTC
I've tested it on SLE-15-SP5.

1. I've added https://download.opensuse.org/repositories/systemsmanagement:/SCC/SLE_15_SP5/ as repo.

2. zypper install --oldpackage suseconnect-ng-1.5.0~git0.b838c19-150500.75.1
rpm -qi suseconnect-ng
Name        : suseconnect-ng
Version     : 1.5.0~git0.b838c19
Release     : 150500.75.1
Architecture: x86_64
Install Date: Mi 17 Jan 2024 14:43:19 CET
Group       : System/Management
Size        : 7381576
License     : LGPL-2.1-or-later
Signature   : RSA/SHA256, Mi 17 Jan 2024 10:42:48 CET, Key ID acaa9cf7e6e5a213
Source RPM  : suseconnect-ng-1.5.0~git0.b838c19-150500.75.1.src.rpm
Build Date  : Mi 17 Jan 2024 10:42:33 CET
Build Host  : i03-ch2b
Relocations : (not relocatable)
Packager    : https://www.suse.com/
Vendor      : obs://build.opensuse.org/systemsmanagement
URL         : https://github.com/SUSE/connect-ng
Summary     : Utility to register a system with the SUSE Customer Center
Description :
This package provides a command line tool for connecting a
client system to the SUSE Customer Center. It will connect the system to your
product subscriptions and enable the product repositories/services locally.
suseconnect-ng reduces the size of its runtime dependencies compared to the
replaced SUSEConnect.
Distribution: systemsmanagement:SCC / SLE_15_SP5

hana22:~ # zypper in -y http://updates.suse.de/SUSE/Updates/SLE-Module-Server-Applications/15-SP3/x86_64/update/noarch/migrate-sles-to-sles4sap-15.1.2-3.9.1.noarch.rpm
It works fine.
Comment 23 Alvaro Carvajal 2024-01-17 16:25:04 UTC
Created attachment 871948 [details]
Screenshot after migration

(In reply to Thomas Schmidt from comment #20)
> We added another fix to handle that case. Please test with the package from
> https://download.opensuse.org/repositories/systemsmanagement:/SCC/ (it has
> ~git0.b838c19 as version).
> 
> I successfully verified it with: 
> 
> $ docker run registry.suse.com/suse/sle15:15.3
> > zypper rm -y container-suseconnect
> > zypper in -y https://download.opensuse.org/repositories/systemsmanagement:/SCC/SLE_15_SP3/x86_64/suseconnect-ng-1.5.0~git0.b838c19-150300.75.1.x86_64.rpm
> > zypper in -y http://updates.suse.de/SUSE/Updates/SLE-Module-Server-Applications/15-SP3/x86_64/update/noarch/migrate-sles-to-sles4sap-15.1.2-3.9.1.noarch.rpm
> > /usr/sbin/Migrate_SLES_to_SLES-for-SAP.sh

Had to install suseconnect-ng-1.5.0~git0.b838c19-150500.75.1.x86_64 with:

zypper install --oldpackage suseconnect-ng-1.5.0~git0.b838c19-150500.75.1.x86_64

As without --oldpackage, zypper complained that the version already installed as part of 15-SP5 QU RC1 Build 134.1 was newer, but after installing it, installing migrate-sles-to-sles4sap and running the migration script, everything worked so it fixed the issue.

Attaching a screenshot after system was migrated.
Comment 26 Maintenance Automation 2024-01-23 08:30:02 UTC
SUSE-RU-2024:0176-1: An update that has two fixes can now be installed.

Category: recommended (critical)
Bug References: 1217961, 1218649
Sources used:
SUSE Linux Enterprise Micro 5.2 (src): suseconnect-ng-1.6.0~git0.31371c8-150300.3.6.1
SUSE Linux Enterprise Micro for Rancher 5.2 (src): suseconnect-ng-1.6.0~git0.31371c8-150300.3.6.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 27 Maintenance Automation 2024-01-23 08:30:04 UTC
SUSE-RU-2024:0175-1: An update that has two fixes can now be installed.

Category: recommended (critical)
Bug References: 1217961, 1218649
Sources used:
SUSE Linux Enterprise Micro 5.1 (src): suseconnect-ng-1.6.0~git0.31371c8-150300.3.11.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 28 Maintenance Automation 2024-01-23 08:30:06 UTC
SUSE-RU-2024:0174-1: An update that has three fixes can now be installed.

Category: recommended (critical)
Bug References: 1217961, 1218364, 1218649
Sources used:
SUSE Linux Enterprise High Performance Computing 12 SP5 (src): suseconnect-ng-1.6.0~git0.31371c8-3.6.1
SUSE Linux Enterprise Server 12 SP5 (src): suseconnect-ng-1.6.0~git0.31371c8-3.6.1
SUSE Linux Enterprise Server for SAP Applications 12 SP5 (src): suseconnect-ng-1.6.0~git0.31371c8-3.6.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 29 Maintenance Automation 2024-01-23 08:30:07 UTC
SUSE-RU-2024:0173-1: An update that has two fixes can now be installed.

Category: recommended (critical)
Bug References: 1217961, 1218649
Sources used:
Basesystem Module 15-SP5 (src): suseconnect-ng-1.6.0~git0.31371c8-150500.3.12.1
openSUSE Leap 15.5 (src): suseconnect-ng-1.6.0~git0.31371c8-150500.3.12.1
SUSE Linux Enterprise Micro 5.5 (src): suseconnect-ng-1.6.0~git0.31371c8-150500.3.12.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 35 Maintenance Automation 2024-01-23 20:30:56 UTC
SUSE-RU-2024:0189-1: An update that has two fixes can now be installed.

Category: recommended (critical)
Bug References: 1217961, 1218649
Sources used:
SUSE Linux Enterprise High Performance Computing 15 SP1 LTSS 15-SP1 (src): suseconnect-ng-1.6.0~git0.31371c8-150100.3.15.1
SUSE Linux Enterprise High Performance Computing 15 SP2 LTSS 15-SP2 (src): suseconnect-ng-1.6.0~git0.31371c8-150100.3.15.1
SUSE Linux Enterprise High Performance Computing LTSS 15 SP3 (src): suseconnect-ng-1.6.0~git0.31371c8-150100.3.15.1
SUSE Linux Enterprise Server 15 SP1 LTSS 15-SP1 (src): suseconnect-ng-1.6.0~git0.31371c8-150100.3.15.1
SUSE Linux Enterprise Server 15 SP2 LTSS 15-SP2 (src): suseconnect-ng-1.6.0~git0.31371c8-150100.3.15.1
SUSE Linux Enterprise Server 15 SP3 LTSS 15-SP3 (src): suseconnect-ng-1.6.0~git0.31371c8-150100.3.15.1
SUSE Linux Enterprise Server for SAP Applications 15 SP1 (src): suseconnect-ng-1.6.0~git0.31371c8-150100.3.15.1
SUSE Linux Enterprise Server for SAP Applications 15 SP2 (src): suseconnect-ng-1.6.0~git0.31371c8-150100.3.15.1
SUSE Linux Enterprise Server for SAP Applications 15 SP3 (src): suseconnect-ng-1.6.0~git0.31371c8-150100.3.15.1
SUSE Enterprise Storage 7.1 (src): suseconnect-ng-1.6.0~git0.31371c8-150100.3.15.1
SUSE CaaS Platform 4.0 (src): suseconnect-ng-1.6.0~git0.31371c8-150100.3.15.1
SUSE Linux Enterprise Server 15 SP1 (src): suseconnect-ng-1.6.0~git0.31371c8-150100.3.15.1
SUSE Linux Enterprise Server 15 SP2 (src): suseconnect-ng-1.6.0~git0.31371c8-150100.3.15.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 36 Maintenance Automation 2024-01-23 20:30:57 UTC
SUSE-RU-2024:0188-1: An update that has two fixes can now be installed.

Category: recommended (critical)
Bug References: 1217961, 1218649
Sources used:
openSUSE Leap 15.4 (src): suseconnect-ng-1.6.0~git0.31371c8-150400.3.22.1
openSUSE Leap Micro 5.3 (src): suseconnect-ng-1.6.0~git0.31371c8-150400.3.22.1
openSUSE Leap Micro 5.4 (src): suseconnect-ng-1.6.0~git0.31371c8-150400.3.22.1
SUSE Linux Enterprise Micro for Rancher 5.3 (src): suseconnect-ng-1.6.0~git0.31371c8-150400.3.22.1
SUSE Linux Enterprise Micro 5.3 (src): suseconnect-ng-1.6.0~git0.31371c8-150400.3.22.1
SUSE Linux Enterprise Micro for Rancher 5.4 (src): suseconnect-ng-1.6.0~git0.31371c8-150400.3.22.1
SUSE Linux Enterprise Micro 5.4 (src): suseconnect-ng-1.6.0~git0.31371c8-150400.3.22.1
SUSE Linux Enterprise High Performance Computing ESPOS 15 SP4 (src): suseconnect-ng-1.6.0~git0.31371c8-150400.3.22.1
SUSE Linux Enterprise High Performance Computing LTSS 15 SP4 (src): suseconnect-ng-1.6.0~git0.31371c8-150400.3.22.1
SUSE Linux Enterprise Desktop 15 SP4 LTSS 15-SP4 (src): suseconnect-ng-1.6.0~git0.31371c8-150400.3.22.1
SUSE Linux Enterprise Server 15 SP4 LTSS 15-SP4 (src): suseconnect-ng-1.6.0~git0.31371c8-150400.3.22.1
SUSE Linux Enterprise Server for SAP Applications 15 SP4 (src): suseconnect-ng-1.6.0~git0.31371c8-150400.3.22.1
SUSE Manager Proxy 4.3 (src): suseconnect-ng-1.6.0~git0.31371c8-150400.3.22.1
SUSE Manager Retail Branch Server 4.3 (src): suseconnect-ng-1.6.0~git0.31371c8-150400.3.22.1
SUSE Manager Server 4.3 (src): suseconnect-ng-1.6.0~git0.31371c8-150400.3.22.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 38 OBSbugzilla Bot 2024-02-15 16:05:05 UTC
This is an autogenerated message for OBS integration:
This bug (1218649) was mentioned in
https://build.opensuse.org/request/show/1146892 Factory / suseconnect-ng
Comment 40 lili zhao 2024-03-15 03:17:18 UTC
Updates:
- Checked 15-SP5 QU test cases, horizontal migration passed
  ppc64le-Build136.1: https://openqa.suse.de/tests/13708463#dependencies
  x86_64-Build136.1:  https://openqa.suse.de/tests/13708624#dependencies

- Also, I am adding horizontal migration test cases to QAM incidents job groups, all the cases are passed.
  15-SP5: https://openqa.suse.de/tests/13764966#dependencies (passed, no LTSS)
  15-SP4: https://openqa.suse.de/tests/13779136#dependencies (passed, LTSS)
  15-SP3: https://openqa.suse.de/tests/13779304#dependencies (passed, LTSS)
  15-SP2: https://openqa.suse.de/tests/13779317#dependencies (passed, LTSS)
  15-SP1: End of LTSS.
  12-SP5: https://openqa.suse.de/tests/13778201#dependencies (passed, no LTSS)

This bug can be moved to "resolved fixed"
Comment 41 Thomas Schmidt 2024-03-15 15:32:43 UTC
Closing
Comment 42 lili zhao 2024-03-18 00:47:55 UTC
Thanks for the fix. Moved to "verified fixed".