Bug 1116524 - Package xen-tools-4.11.0_09-2.1.x86_64 broken: Missing /bin/domu-xenstore
Package xen-tools-4.11.0_09-2.1.x86_64 broken: Missing /bin/domu-xenstore
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Xen
Current
x86-64 Other
: P5 - None : Major (vote)
: ---
Assigned To: Xen Virtualization
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-11-19 09:48 UTC by Olaf Martens
Modified: 2018-12-28 11:44 UTC (History)
4 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Olaf Martens 2018-11-19 09:48:02 UTC
After upgrading to this version of xen-tools in the dom0 the virtual networking started to misbehave. As it is of now, the vif scripts (here it's /etc/xen/scripts/vif-bridge) don't find the network interfaces they are supposed to bring up. Instead an error message is output to /var/log/messages:

2018-11-19T09:39:07.033763+01:00 gagazet root: /etc/xen/scripts/vif-bridge: online type_if=vif XENBUS_PATH=backend/vif/6/0
2018-11-19T09:39:07.104544+01:00 gagazet root: /etc/xen/scripts/vif-bridge: No device details in backend/vif/6/0, exiting.
2018-11-19T09:39:07.226761+01:00 gagazet root: /etc/xen/scripts/vif-bridge: add type_if=tap XENBUS_PATH=backend/vif/6/0
2018-11-19T09:39:07.354079+01:00 gagazet root: /etc/xen/scripts/vif-bridge: No device details in /local/domain/0/backend/vif/6/0, exiting.

As it is, after some digging through the fs I found that the tool /bin/domu-xenstore is missing (attempting to locate domu-xenstore anywhere within the fs was met with failure), which in turn means that various symlinks in /usr/bin are invalid because of the missing target (the symlinks in question are marked yellow on a red background in the dir listing). This in turn means that the scripts cannot obtain any data crucial to their operation.

As a workaround, the virtual interface may be attached to the bridge and brought up manually, but that's highly unsatisfactory.

This can be reproduced by simply upgrading any Tumbleweed prior to 20181116-0 to 20181116-0 and also upgrading xen-tools to 4.11.0_09-2.1 in the process.
Comment 1 Dario Faggioli 2018-11-20 18:04:02 UTC
(In reply to Olaf Martens from comment #0)
> After upgrading to this version of xen-tools in the dom0 the virtual
> networking started to misbehave. As it is of now, the vif scripts (here it's
> /etc/xen/scripts/vif-bridge) don't find the network interfaces they are
> supposed to bring up. Instead an error message is output to
> /var/log/messages:
> 
> [...]
>
> As it is, after some digging through the fs I found that the tool
> /bin/domu-xenstore is missing (attempting to locate domu-xenstore anywhere
> within the fs was met with failure), which in turn means that various
> symlinks in /usr/bin are invalid because of the missing target (the symlinks
> in question are marked yellow on a red background in the dir listing). This
> in turn means that the scripts cannot obtain any data crucial to their
> operation.
>
Mmm... but why this domu-xenstore tool should be available in dom0 in the first place?

I don't have it, in any dom0 --either on SLES or openSUSE box. Doing `cnf domu-xenstore' tells me that it comes from the "xen-tools-domU" package, which is not something one should have installed in dom0.

In fact, if you try to install it, there is a conflict (with xen-tools and patterns-server-xen-server packages).

> This can be reproduced by simply upgrading any Tumbleweed prior to
> 20181116-0 to 20181116-0 and also upgrading xen-tools to 4.11.0_09-2.1 in
> the process.

Ok, I have an updated Tumbleweed, and I'll look into this. But whatever the issue is, I don't think it's related to the lack of domu-xenstore.
Comment 2 Dario Faggioli 2018-11-21 04:06:43 UTC
Ah, I've tried on an update Tumbleweed, and I also see this, checking
`journalctl -u libvirtd':

/etc/xen/scripts/vif-bridge: offline type_if=vif XENBUS_PATH=backend/vif/7/0
/etc/xen/scripts/vif-bridge: No device details in backend/vif/7/0, exiting

Checking vif-bridge, the above comes from here:

mac=$(xenstore_read_default "$XENBUS_PATH/mac" "")
if [ -z "$mac" ]
then
    log debug "No device details in $XENBUS_PATH, exiting."
    exit 0
fi

I've therefore went checking what's about backend/vif/7/0 in the output of xenstore-ls, but, surprise, there's no xenstore-ls command. In fact:

lrwxrwxrwx 1 root root    18 13 nov 17.38 /usr/bin/xenstore -> /bin/domu-xenstore
lrwxrwxrwx 1 root root     8 13 nov 17.34 /usr/bin/xenstore-chmod -> xenstore
-rwxr-xr-x 1 root root 14576 13 nov 17.38 /usr/bin/xenstore-control
lrwxrwxrwx 1 root root     8 13 nov 17.34 /usr/bin/xenstore-exists -> xenstore
lrwxrwxrwx 1 root root     8 13 nov 17.34 /usr/bin/xenstore-list -> xenstore
lrwxrwxrwx 1 root root     8 13 nov 17.34 /usr/bin/xenstore-ls -> xenstore
lrwxrwxrwx 1 root root     8 13 nov 17.34 /usr/bin/xenstore-read -> xenstore
lrwxrwxrwx 1 root root     8 13 nov 17.34 /usr/bin/xenstore-rm -> xenstore
lrwxrwxrwx 1 root root     8 13 nov 17.34 /usr/bin/xenstore-watch -> xenstore
lrwxrwxrwx 1 root root     8 13 nov 17.34 /usr/bin/xenstore-write -> xenstore

So this is, apparently, from where the mentioning of domu-xenstore comes. So, I don't think the problem is that domu-xenstore went missing... but indeed it looks to me like we're missing something!

Charles?
Comment 3 Dario Faggioli 2018-11-21 04:14:41 UTC
E.g., downloading the Leap 15 package from here https://software.opensuse.org/package/xen-tools , and doing `rpm -qlpv' on it, shows me this:

-rwxr-xr-x    1 root    root                    19216 ago 17 14:19 /usr/bin/xenstore
lrwxrwxrwx    1 root    root                        8 ago 17 14:15 /usr/bin/xenstore-chmod -> xenstore
-rwxr-xr-x    1 root    root                    10488 ago 17 14:19 /usr/bin/xenstore-control
lrwxrwxrwx    1 root    root                        8 ago 17 14:15 /usr/bin/xenstore-exists -> xenstore
lrwxrwxrwx    1 root    root                        8 ago 17 14:15 /usr/bin/xenstore-list -> xenstore
lrwxrwxrwx    1 root    root                        8 ago 17 14:15 /usr/bin/xenstore-ls -> xenstore
lrwxrwxrwx    1 root    root                        8 ago 17 14:15 /usr/bin/xenstore-read -> xenstore
lrwxrwxrwx    1 root    root                        8 ago 17 14:15 /usr/bin/xenstore-rm -> xenstore
lrwxrwxrwx    1 root    root                        8 ago 17 14:15 /usr/bin/xenstore-watch -> xenstore
lrwxrwxrwx    1 root    root                        8 ago 17 14:15 /usr/bin/xenstore-write -> xenstore
Comment 4 Charles Arnold 2018-11-21 22:56:03 UTC
The problem showed up when I accepted fixes for creating reproducible
builds (see bugs boo#1047218, boo#1062303)

The patch tmp_build.patch was modified this way,

-+      $(CC) $< $(CFLAGS) $(LDFLAGS) -Wl,--build-id=uuid -L. -lxenstore $(LDLIBS_libxentoolcore) $(SOCKET_LIBS) -o domu-$@
++      $(CC) $< $(CFLAGS) $(LDFLAGS) -Wl,--build-id=sha1 -L. -lxenstore $(LDLIBS_libxentoolcore) $(SOCKET_LIBS) -o domu-$@

and it appears that this change is what is breaking the links.


Olaf,

Could you please try the RPMs from the Virtualization/xen repo and see
if they resolve the problem? Minimally you will need to install xen,
xen-tools, and xen-libs.
Comment 5 Dario Faggioli 2018-11-22 11:53:57 UTC
(In reply to Charles Arnold from comment #4)
> Olaf,
> 
> Could you please try the RPMs from the Virtualization/xen repo and see
> if they resolve the problem? Minimally you will need to install xen,
> xen-tools, and xen-libs.
>
Ok, so, I added this repository https://download.opensuse.org/repositories/Virtualization/openSUSE_Factory/ , installed xen, xen-libs and xen-tools packages from there, and now things work for me.

Dario
Comment 6 Charles Arnold 2018-12-03 22:28:39 UTC
This fix is now in Factory. It should show up in a Tumbleweed build soon.
Comment 7 Bernhard Wiedemann 2018-12-27 17:12:26 UTC
I think, the problem triggered by my change
was coming from this section in xen.spec:

# Create symlinks for keymaps
%fdupes -s $RPM_BUILD_ROOT

This erroneously created cross-package symlinks between
xen-tools /usr/bin/xenstore and
xen-tools-domU /bin/domu-xenstore
that became dangling because one package does not require the other.

Dropping the fdups call fixes this bug and produces these:
 xen-tools.x86_64: W: files-duplicate /usr/lib64/python3.6/site-packages/xen/__init__.py /usr/lib64/python3.6/site-packages/xen/lowlevel/__init__.py
 xen-doc-html.x86_64: W: files-duplicate /usr/share/doc/packages/xen/html/misc/vtpm-platforms.txt /usr/share/doc/packages/xen/misc/vtpm-platforms.txt
 xen-doc-html.x86_64: W: files-duplicate /usr/share/doc/packages/xen/misc/crashdb.txt /usr/share/doc/packages/xen/html/misc/crashdb.txt
 xen-doc-html.x86_64: W: files-duplicate /usr/share/doc/packages/xen/misc/xenpaging.txt /usr/share/doc/packages/xen/html/misc/xenpaging.txt

I think at least 3 of 4 should not be duplicated in the first place.
And hardlinks would probably be better anyway.
Comment 8 Bernhard Wiedemann 2018-12-28 11:44:12 UTC
submitted https://build.opensuse.org/request/show/661659
test with care