Bug 1214476 - Xen VM cannot start libxenlight failed (after snapshot 20230821)
Summary: Xen VM cannot start libxenlight failed (after snapshot 20230821)
Status: RESOLVED INVALID
Alias: None
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Xen (show other bugs)
Version: Current
Hardware: x86-64 openSUSE Tumbleweed
: P5 - None : Major (vote)
Target Milestone: ---
Assignee: Xen Virtualization
QA Contact: E-mail List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-08-22 19:10 UTC by Philippe Condé
Modified: 2023-08-23 16:49 UTC (History)
3 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments
journalctl with start virtmgr and start vm (7.26 KB, text/plain)
2023-08-22 19:10 UTC, Philippe Condé
Details
XML define VM opensustumbleed (2.06 KB, text/plain)
2023-08-23 08:44 UTC, Philippe Condé
Details
log file qemu-dm-opensusetumbleweed.log (295 bytes, text/x-log)
2023-08-23 08:46 UTC, Philippe Condé
Details
log file /var/log/libvirt/libxl/libxl-driver.log (16.18 KB, text/x-log)
2023-08-23 08:49 UTC, Philippe Condé
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Philippe Condé 2023-08-22 19:10:33 UTC
Created attachment 868948 [details]
journalctl with start virtmgr and start vm

I have a server with tumbleweed wher I have one VM. I cannot start it again after snapshot 202308321. I receive this error in virtmgr

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 72, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 108, in tmpcb
    callback(*args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 57, in newfn
    ret = fn(self, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/virt-manager/virtManager/object/domain.py", line 1425, in startup
    self._backend.create()
  File "/usr/lib64/python3.11/site-packages/libvirt.py", line 1373, in create
    raise libvirtError('virDomainCreate() failed')
libvirt.libvirtError: internal error: libxenlight failed to create new domain 'opensusetumbleweed'


Journalctl output in attach

This vm worked since months without problem.
Comment 1 James Fehlig 2023-08-22 19:53:15 UTC
There's nothing helpful in the journal output. Can you provide the VM config (e.g. 'virsh dumpxml vm-name'), /var/log/libvirt/libxl/libxl-driver.log, and /var/log/xen/qemu-dm-<vm-name>.log? Thanks!
Comment 2 Philippe Condé 2023-08-23 08:44:28 UTC
Created attachment 868960 [details]
XML define VM opensustumbleed

Output or virsh dumpxml opensusetumbleweed
Comment 3 Philippe Condé 2023-08-23 08:46:12 UTC
Created attachment 868961 [details]
log file qemu-dm-opensusetumbleweed.log
Comment 4 Philippe Condé 2023-08-23 08:49:08 UTC
Created attachment 868962 [details]
log file /var/log/libvirt/libxl/libxl-driver.log
Comment 5 Philippe Condé 2023-08-23 09:21:02 UTC
Hello,


I found the problem. 
in the xml définition of opensustumbleweed there was
    <disk type="file" device="cdrom">
      <driver name="qemu" type="raw"/>
      <source file="/local/download64/OpenSuse_iso/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20211129-Media.iso"/>
      <target dev="hdb" bus="ide"/>
      <readonly/>
      <address type="drive" controller="0" bus="0" target="0" unit="1"/>
    </disk>
 

But I downloaded a new iso snapshot and removed the old. Changing in the xml to the new snapshot solved the problem.


Question: how can I change this cdrom définition so that it goes to real cdrom
I have in /dev
hpprol2:/dev # ls -ls /dev/cdrom
0 lrwxrwxrwx 1 root root 3 Aug 23 10:57 /dev/cdrom -> sr0
hpprol2:/dev # ls -l sr0 
brw-rw----+ 1 root cdrom 11, 0 Aug 23 10:57 sr0

Do I need to remove the source file or update it to /dev/cdrom?

Many thanks in advance
Philippe
Comment 6 Santiago Zarate 2023-08-23 09:55:38 UTC
(In reply to Philippe Condé from comment #5)
> Hello,
> 
> 
> I found the problem. 
> in the xml définition of opensustumbleweed there was
>     <disk type="file" device="cdrom">
>       <driver name="qemu" type="raw"/>
>       <source
> file="/local/download64/OpenSuse_iso/openSUSE-Tumbleweed-DVD-x86_64-
> Snapshot20211129-Media.iso"/>
>       <target dev="hdb" bus="ide"/>
>       <readonly/>
>       <address type="drive" controller="0" bus="0" target="0" unit="1"/>
>     </disk>
>  
> 
> But I downloaded a new iso snapshot and removed the old. Changing in the xml
> to the new snapshot solved the problem.
> 
> 
> Question: how can I change this cdrom définition so that it goes to real
> cdrom
> I have in /dev
> hpprol2:/dev # ls -ls /dev/cdrom
> 0 lrwxrwxrwx 1 root root 3 Aug 23 10:57 /dev/cdrom -> sr0
> hpprol2:/dev # ls -l sr0 
> brw-rw----+ 1 root cdrom 11, 0 Aug 23 10:57 sr0
> 
> Do I need to remove the source file or update it to /dev/cdrom?
> 
> Many thanks in advance
> Philippe

You can take a look at the documentation, https://libvirt.org/formatdomain.html#hard-drives-floppy-disks-cdroms

but `<source dev='/dev/cdrom'/>` should do the trick.
Comment 7 Santiago Zarate 2023-08-23 09:57:46 UTC
(In reply to James Fehlig from comment #1)
> There's nothing helpful in the journal output. Can you provide the VM config
> (e.g. 'virsh dumpxml vm-name'), /var/log/libvirt/libxl/libxl-driver.log, and
> /var/log/xen/qemu-dm-<vm-name>.log? Thanks!

is there a way to make the journal provide such output directly or improve the message to the user? sometimes in QA we have issues with the virt backend, and unless one knows where to look, initially one is in the dark.
Comment 8 Philippe Condé 2023-08-23 10:45:05 UTC
(In reply to Santiago Zarate from comment #6)

> 
> You can take a look at the documentation,
> https://libvirt.org/formatdomain.html#hard-drives-floppy-disks-cdroms
> 
> but `<source dev='/dev/cdrom'/>` should do the trick.

Many Thanks, 
I stopped the VM, changed the définition in virtmgr for the cdrom and restarted the VM.All seems correct I can log in and see that the cdrom is now present.

For me this bug report can be closed

Regards
Philippe
Comment 9 James Fehlig 2023-08-23 16:49:52 UTC
(In reply to Santiago Zarate from comment #7)
> (In reply to James Fehlig from comment #1)
> > There's nothing helpful in the journal output. Can you provide the VM config
> > (e.g. 'virsh dumpxml vm-name'), /var/log/libvirt/libxl/libxl-driver.log, and
> > /var/log/xen/qemu-dm-<vm-name>.log? Thanks!
> 
> is there a way to make the journal provide such output directly or improve
> the message to the user? sometimes in QA we have issues with the virt
> backend, and unless one knows where to look, initially one is in the dark.

Admittedly with Xen it's just a matter of knowing where to look. libvirt (in this case virtxend) sends all log messages to the journal. With some work, messages from Xen's libxl library could be sent to the journal in addition to /var/log/libvirt/libxl/libxl-driver.log. But that wouldn't have helped in this case where the actual problem was logged to /var/log/xen/qemu-dm-opensusetumbleweed.log

qemu-system-i386: -drive if=none,readonly=on,id=ide-832,file=/local/download64/OpenSuse_iso/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20211129-Media.iso,format=raw: Could not open '/local/download64/OpenSuse_iso/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20211129-Media.iso': No such file or directory

In most cases Xen spawns a qemu process for each VM to handle device emulation, among other things. I'm not familiar enough with Xen to estimate the effort needed to make these qemu processes send their log messages to the journal, or if that's even desirable. Would need consultation with the Xen community. But it's out of scope for this bug report.