Bug 1219818

*** Bug 1219838 has been marked as a duplicate of this bug. ***

This bug has a negative impact on our build service. Bumping priority. Please, have a look.

actually: also broken on aarch64 and powerpc (tested both power8 and power9).
in 15-sp6 kvm currently only works on x86.

(In reply to Ruediger Oertel from comment #3)
> actually: also broken on aarch64 and powerpc (tested both power8 and power9).
> in 15-sp6 kvm currently only works on x86.
>
Mmm... weird.

Is it possible to try with Tumbleweed? I'd do it myself, but I'm not finding any usable aarch64 (and even less s390 and ppc ones) machine...

the TW version where qemu is built from identical source works, it's really something between qemu and kernel in 15-SP6

Please provide access to a test machine for s390x with the packages installed.

Having supportconfig etc could possibly help, as well as the list of packages installed and a lot more information.

"qemu-s390x-8.2.0 is failing with qemu-system-s390x: invalid accelerator kvm"

is not telling much on its own,

we need to know more about why that happens, whether /dev/kvm is available at all in this environment, etc,

and the information present in this bug does not provide all the necessary contextual information.

Therefore either actionable debugging information needs to be provided, or access to the system is required.

Thanks,

CLaudio

Can you provide access to the machine or it's impossible as it is automated?

In case it's impossible I can most probably install a machine, I just need to know if it's qemu running on s390 zVM or on top of KVM.

(In reply to Marcela Maslanova from comment #7)
> Can you provide access to the machine or it's impossible as it is automated?
> 
> In case it's impossible I can most probably install a machine, I just need
> to know if it's qemu running on s390 zVM or on top of KVM.

Hi Marcela,

I am not sure about the OpenQA automation.

If this specific OpenQA host is not accessible, then preparing a new machine / s390x slice makes sense.

I looked at the pretty pictures and found out that at some point there is this command that is ran (I manually retyped the command):

qemu-system-s390x -nographic -enable-kvm -kernel /boot/image -initrd /boot/initrd

If you can prepare an s390x zVM slice, with the specific /boot/image and /boot/initrd binaries that fail to run and give me access I can start taking a look.

Thanks,

C

Please beware that some architectures/platforms including aarch64 have restrictions on nested virtualization (Armv8.3-A and later), so trying to use KVM inside a KVM based openQA worker is expected to fail.

(In reply to Andreas Färber from comment #10)
> Please beware that some architectures/platforms including aarch64 have
> restrictions on nested virtualization (Armv8.3-A and later), so trying to
> use KVM inside a KVM based openQA worker is expected to fail.

We also never tested nested S390x KVM in a Slice, I have no idea if it works.

But again, we don't know because we only have the OpenQA pictures to look at.

I did get the "invalid accelerator kvm" on aarch64 bare metal
and on power9 bare metal and s390x LPAR, no real nested virtualisation
involved and /dev/kvm was present as well.

I can try to deploy an sp6 image on one machine tomorrow and run supportconfig
if that helps, but the build workers are in their separate network, no direct
access possible.

Should be solved by: https://build.suse.de/request/show/321910

A SR mentioning this bug was successfully integrated into Build 58.1. Please, if needed, set it to RESOLVED FIXED for QE to verify it.

s390x is not part of 58.1 build, I will resolve it with next build for s390x

*** Bug 1220321 has been marked as a duplicate of this bug. ***

*** Bug 1220311 has been marked as a duplicate of this bug. ***

== Comment: #0 - Sean S. Safron <safron@us.ibm.com> - 2024-02-22 09:36:35 ==
---Problem Description---
KVM guests cannot use domain type "kvm" on SLES15-SP6 snapshot-202402-1

Running `virt-install` causes a guest to get created with <domain type='qemu'>. Trying to edit the guest XML to <domain type='kvm'> results in this error:

error: unsupported configuration: Emulator '/usr/bin/qemu-system-s390x' does not support virt type 'kvm'

Otherwise the guest comes up and mostly runs fine (though the bootup is very slow - it takes 5-10 minutes to get to a login prompt)

I also noticed "-accel tcg" in /var/log/libvirt/qemu/<guest>.log. It is normally set to "kvm".

This problem was not seen in SLES15-SP6-beta3 build (an earlier build).

KVM Host System Info:
za93ka:~ # uname -a
Linux za93ka 6.4.0-150600.7-default #1 SMP Thu Feb  8 20:18:21 UTC 2024 (eafe2c2) s390x s390x s390x GNU/Linux
za93ka:~ # cat /etc/os-release
NAME="SLES"
VERSION="15-SP6"
VERSION_ID="15.6"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP6"
ID="sles"
ID_LIKE="suse"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:15:sp6"
DOCUMENTATION_URL="https://documentation.suse.com/"
za93ka:~ # qemu-system-s390x -version
QEMU emulator version 8.2.0 (SUSE Linux Enterprise 15)
Copyright (c) 2003-2023 Fabrice Bellard and the QEMU Project developers
za93ka:~ # virsh -v
10.0.0

Contact Information = Sean Safron. safron@us.ibm.com

---Steps to Reproduce---
1. Confirm kvm module is loaded:

za93ka:~ # lsmod |grep kvm
kvm                   483328  0

2. Run virt-install to create a guest. For example:

virt-install --connect qemu:///system --name za93ka-sanity-provision-micro-qcow-10-20-112-43 --iothreads  1  --metadata description=\\"qcow:/guestimages0/za93ka-sanity-provision-micro-qcow-10-20-112-43/za93ka-sanity-provision-micro-qcow-10-20-112-43.qcow2\\" --vcpus 1 --memory 2048 --disk za93ka-sanity-provision-micro-qcow-10-20-112-43.qcow2,cache=,io=,driver.iothread=1 --disk path=seed.iso,device=cdrom,bus=scsi --network type=direct,source=vlan1307,source.mode=bridge --boot hd --noautoconsole --console log.file=/guestimages0/za93ka-sanity-provision-micro-qcow-10-20-112-43/console.log,log.append=yes

3. See that the resulting guest XML contains <domain type='qemu'>

za93ka:~ # virsh dumpxml za93ka-sanity-provision-micro-qcow-10-20-112-43 |head -1
<domain type='qemu' id='2'>

4. Attempting to edit the XML to <domain type='kvm'> results in this error:

error: unsupported configuration: Emulator '/usr/bin/qemu-system-s390x' does not support virt type 'kvm'

---Debugger---
A debugger is not configured

Machine Type = Type: 3931, Model: 708 A01

---uname output---
Linux za93ka 6.4.0-150600.7-default #1 SMP Thu Feb  8 20:18:21 UTC 2024 (eafe2c2) s390x s390x s390x GNU/Linux

*Additional Instructions for Sean Safron. safron@us.ibm.com:
-Post a private note with access information to the machine that the bug is occuring on.

== Comment: #4 - Thomas Staudt <tstaudt@de.ibm.com> - 2024-02-26 00:51:58 ==

== Comment: #5 - Thomas Staudt <tstaudt@de.ibm.com> - 2024-02-26 00:52:41 ==

== Comment: #6 - Thomas Staudt <tstaudt@de.ibm.com> - 2024-02-26 00:53:22 ==

------- Comment From tstaudt@de.ibm.com 2024-02-26 02:24 EDT-------
Regarding: "This problem was not seen in SLES15-SP6-beta3 build (an earlier build)."

=> Beta3 still had qemu 8.1.0, the snapshot is the first deliverable with qemu 8.2.0

------- Comment From Nina.Schoetterl-Glausch@ibm.com 2024-02-28 08:02 EDT-------
I checked and qemu 8.2.1 from openSUSE fixes the issue https://build.opensuse.org/package/show/Virtualization/qemu
When will that make it our way?
(In reply to LTC BugProxy from comment #5)
> ------- Comment From Nina.Schoetterl-Glausch@ibm.com 2024-02-28 08:02
> EDT-------
> I checked and qemu 8.2.1 from openSUSE fixes the issue
> https://build.opensuse.org/package/show/Virtualization/qemu
> When will that make it our way?
Yes, sorry for that, there was a mistake in the spec file. Thanks also for checking Factory and confirming that it is the same issue.

The updated package is already part of our 15-SP6 repository (since ~11 days, as far as I can see in IBS). I don't know when the next update/milestone/release is planned, but I guess that, whenever that is, it will include the fixed package.
In fact, this is a duplicate of bug 1219818. Marking as such

*** This bug has been marked as a duplicate of bug 1219818 ***

(In reply to LTC BugProxy from comment #21)
> == Comment: #0 - Sean S. Safron <safron@us.ibm.com> - 2024-02-22 09:36:35 ==
> ---Problem Description---
> KVM guests cannot use domain type "kvm" on SLES15-SP6 snapshot-202402-1
> 
> Running `virt-install` causes a guest to get created with <domain
> type='qemu'>. Trying to edit the guest XML to <domain type='kvm'> results in
> this error:
> 
> error: unsupported configuration: Emulator '/usr/bin/qemu-system-s390x' does
> not support virt type 'kvm'
> 
> Otherwise the guest comes up and mostly runs fine (though the bootup is very
> slow - it takes 5-10 minutes to get to a login prompt)
> 
> I also noticed "-accel tcg" in /var/log/libvirt/qemu/<guest>.log. It is
> normally set to "kvm".
> 
> This problem was not seen in SLES15-SP6-beta3 build (an earlier build).
> 
> KVM Host System Info:
> za93ka:~ # uname -a
> Linux za93ka 6.4.0-150600.7-default #1 SMP Thu Feb  8 20:18:21 UTC 2024
> (eafe2c2) s390x s390x s390x GNU/Linux
> za93ka:~ # cat /etc/os-release
> NAME="SLES"
> VERSION="15-SP6"
> VERSION_ID="15.6"
> PRETTY_NAME="SUSE Linux Enterprise Server 15 SP6"
> ID="sles"
> ID_LIKE="suse"
> ANSI_COLOR="0;32"
> CPE_NAME="cpe:/o:suse:sles:15:sp6"
> DOCUMENTATION_URL="https://documentation.suse.com/"
> za93ka:~ # qemu-system-s390x -version
> QEMU emulator version 8.2.0 (SUSE Linux Enterprise 15)
> Copyright (c) 2003-2023 Fabrice Bellard and the QEMU Project developers
> za93ka:~ # virsh -v
> 10.0.0
> 
> Contact Information = Sean Safron. safron@us.ibm.com
> 
> ---Steps to Reproduce---
> 1. Confirm kvm module is loaded:
> 
> za93ka:~ # lsmod |grep kvm
> kvm                   483328  0
> 
> 2. Run virt-install to create a guest. For example:
> 
> virt-install --connect qemu:///system --name
> za93ka-sanity-provision-micro-qcow-10-20-112-43 --iothreads  1  --metadata
> description=\\"qcow:/guestimages0/za93ka-sanity-provision-micro-qcow-10-20-
> 112-43/za93ka-sanity-provision-micro-qcow-10-20-112-43.qcow2\\" --vcpus 1
> --memory 2048 --disk
> za93ka-sanity-provision-micro-qcow-10-20-112-43.qcow2,cache=,io=,driver.
> iothread=1 --disk path=seed.iso,device=cdrom,bus=scsi --network
> type=direct,source=vlan1307,source.mode=bridge --boot hd --noautoconsole
> --console
> log.file=/guestimages0/za93ka-sanity-provision-micro-qcow-10-20-112-43/
> console.log,log.append=yes
> 
> 3. See that the resulting guest XML contains <domain type='qemu'>
> 
> za93ka:~ # virsh dumpxml za93ka-sanity-provision-micro-qcow-10-20-112-43
> |head -1
> <domain type='qemu' id='2'>
> 
> 4. Attempting to edit the XML to <domain type='kvm'> results in this error:
> 
> error: unsupported configuration: Emulator '/usr/bin/qemu-system-s390x' does
> not support virt type 'kvm'
> 
> ---Debugger---
> A debugger is not configured
> 
> Machine Type = Type: 3931, Model: 708 A01
> 
> ---uname output---
> Linux za93ka 6.4.0-150600.7-default #1 SMP Thu Feb  8 20:18:21 UTC 2024
> (eafe2c2) s390x s390x s390x GNU/Linux
> 
> *Additional Instructions for Sean Safron. safron@us.ibm.com:
> -Post a private note with access information to the machine that the bug is
> occuring on.
> 
> == Comment: #4 - Thomas Staudt <tstaudt@de.ibm.com> - 2024-02-26 00:51:58 ==
> 
> == Comment: #5 - Thomas Staudt <tstaudt@de.ibm.com> - 2024-02-26 00:52:41 ==
> 
> == Comment: #6 - Thomas Staudt <tstaudt@de.ibm.com> - 2024-02-26 00:53:22 ==
> 
> ------- Comment From tstaudt@de.ibm.com 2024-02-26 02:24 EDT-------
> Regarding: "This problem was not seen in SLES15-SP6-beta3 build (an earlier
> build)."
> 
> => Beta3 still had qemu 8.1.0, the snapshot is the first deliverable with
> qemu 8.2.0
> 
> ------- Comment From Nina.Schoetterl-Glausch@ibm.com 2024-02-28 08:02
> EDT-------
> I checked and qemu 8.2.1 from openSUSE fixes the issue
> https://build.opensuse.org/package/show/Virtualization/qemu
> When will that make it our way?
> (In reply to LTC BugProxy from comment #5)
> > ------- Comment From Nina.Schoetterl-Glausch@ibm.com 2024-02-28 08:02
> > EDT-------
> > I checked and qemu 8.2.1 from openSUSE fixes the issue
> > https://build.opensuse.org/package/show/Virtualization/qemu
> > When will that make it our way?
> Yes, sorry for that, there was a mistake in the spec file. Thanks also for
> checking Factory and confirming that it is the same issue.
> 
> The updated package is already part of our 15-SP6 repository (since ~11
> days, as far as I can see in IBS). I don't know when the next
> update/milestone/release is planned, but I guess that, whenever that is, it
> will include the fixed package.
> In fact, this is a duplicate of bug 1219818. Marking as such
> 
> *** This bug has been marked as a duplicate of bug 1219818 ***

Err... There's no need to copy all the comments from the other bug here. If they're marked as duplicates, it means that:
 - they're the same issue
 - we have all the links to the bugs marked as dubplicate already, if we need to access the info from there
 - in this specific case, the bug is already solved

Sorry for the noise, that's what the bugzilla bridge does when a mirrored bug is marked as Duplicate.
It links the new pair of bugs and copies the comments it only finds in either the IBM or SUSE bug.

(In reply to Thomas Staudt from comment #23)
> Sorry for the noise, that's what the bugzilla bridge does when a mirrored
> bug is marked as Duplicate.
> It links the new pair of bugs and copies the comments it only finds in
> either the IBM or SUSE bug.
>
Ok, fine... And sorry, I didn't recall it does that.

it should be fixed in 62.1, public beta.

update state (fixed by Dario in the original bug of which this is a duplicate).

Created attachment 873324 [details]
RPMs installed on the kvm host

Created attachment 873325 [details]
/var/log/libvirt/qemu/<guest>.log

Created attachment 873326 [details]
Guest XML

Verified/Fixed with Build14.26.

------- Comment From safron@us.ibm.com 2024-03-20 08:42 EDT-------
I tested with sle-15-sp6-full-publicbeta-202403 and verified that I can create a guest with domain type='kvm' and it runs fine.

za93kb:~ # uname -a
Linux za93kb 6.4.0-150600.9-default #1 SMP Fri Feb 23 21:11:52 UTC 2024 (375d88d) s390x s390x s390x GNU/Linux
za93kb:~ # cat /etc/os-release
NAME="SLES"
VERSION="15-SP6"
VERSION_ID="15.6"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP6"
ID="sles"
ID_LIKE="suse"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:15:sp6"
DOCUMENTATION_URL="https://documentation.suse.com/"
za93kb:~ # zypper repos
Repository priorities are without effect. All enabled repositories share the same priority.

#  | Alias                              | Name                              | Enabled | GPG Check | Refresh
---+------------------------------------+-----------------------------------+---------+-----------+--------
1 | Basesystem-Module_15.6-0           | sle-module-basesystem             | Yes     | (r ) Yes  | Yes
2 | Desktop-Applications-Module_15.6-0 | sle-module-desktop-applications   | Yes     | (r ) Yes  | Yes
3 | Development-Tools-Module_15.6-0    | sle-module-development-tools      | Yes     | (r ) Yes  | Yes
4 | Python-3-Module_15.6-0             | sle-module-python3                | Yes     | (r ) Yes  | Yes
5 | SLES15-SP6-15.6-0                  | SLES15-SP6-15.6-0                 | Yes     | (r ) Yes  | Yes
6 | Server-Applications-Module_15.6-0  | sle-module-server-applications    | Yes     | (r ) Yes  | Yes
7 | xyodi-Module-Basesystem            | sle-15-sp6-full-publicbeta-202403 | Yes     | (r ) Yes  | Yes
8 | xyodi-Module-Containers            | sle-15-sp6-full-publicbeta-202403 | Yes     | (r ) Yes  | Yes
9 | xyodi-Module-Development-Tools     | sle-15-sp6-full-publicbeta-202403 | Yes     | (r ) Yes  | Yes
10 | xyodi-Module-Public-Cloud          | sle-15-sp6-full-publicbeta-202403 | Yes     | (r ) Yes  | Yes
11 | xyodi-Module-Server-Applications   | sle-15-sp6-full-publicbeta-202403 | Yes     | (r ) Yes  | Yes

za93kb:~ # virsh list
Id   Name                                              State
-----------------------------------------------------------------
1    za93kb-external-disk-large-qcow-10-20-101-16      running
2    za93kb-external-disk-large-qcow-10-20-101-22      running
3    za93kbseg-external-disk-large-qcow-10-20-101-24   running
4    za93kbseg-external-disk-large-qcow-10-20-101-25   running

za93kb:~ # virsh dumpxml za93kbseg-external-disk-large-qcow-10-20-101-24 | head -1
<domain type='kvm' id='3'>