Bug 1219766 - Virtualbox USB passthrough fails
Summary: Virtualbox USB passthrough fails
Status: RESOLVED FIXED
Alias: None
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Virtualization:Other (show other bugs)
Version: Current
Hardware: x86-64 openSUSE Tumbleweed
: P5 - None : Normal (vote)
Target Milestone: ---
Assignee: systemd maintainers
QA Contact: E-mail List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-02-09 10:58 UTC by Pinkernell
Modified: 2024-04-06 06:33 UTC (History)
13 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments
Virtual Machine (44.39 KB, application/x-bzip2)
2024-02-12 11:35 UTC, Larry Finger
Details
Trial VirtualBox Machine (153.55 KB, application/x-bzip2)
2024-02-12 12:40 UTC, Larry Finger
Details
vboxdrv kernel crash (7.86 KB, text/plain)
2024-02-12 13:10 UTC, Ingo Göppert
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Pinkernell 2024-02-09 10:58:46 UTC
This is on 

OS: openSUSE Tumbleweed 20240207
Kernel: 6.7.4-1-default (64-bit)

with

Virtualbox 7.0.14-Virt.1699.733.2-x.86_64
Kernel module 7.0.14_k6.7.4_1-Virt.1699.733.2
Guest Windows 10, Guest Additions 7.0.14r161095
USB-3.0 xHCI Controller activated

VBox host software recognises USB device (ScanSnap Scanner) and allows setting device filter. But guest does not find device. Hovering the mouse on USB icon of host software shows "No USB Devices connected". In the list of USB devices that opens after clicking on the icon, the Scanner is blanked out.

This seems new since upgrade to 6.7.2(?) kernel.

Which part of the log do you need? I am new here and happy to help.
Comment 1 Mario Guzman 2024-02-09 21:16:13 UTC
This happened to me after Sundays big update with new kernel and Viertualbox 7.0.14. I tried going back to previous kernel and Virtualbox 7.0.12 but no good. Today I updated to kernel 6.7.4-1 with updates and still nogo. This is a big problem since USB devices no longer work in guests after the TW updates.

Others are reporting this:
https://forums.opensuse.org/t/usb-grayed-out-cant-add-usb-devices-in-virtualbox/172220/53
Comment 2 Larry Finger 2024-02-09 22:36:58 UTC
The problem is with glibc 2.38.

As a work-around, add the repo at https://cdn.opensuse.org/slowroll/repo/oss/ and name it Slowroll. From it, you will be able to use the "Versions" tab and roll back glibc to 3.28. Be careful - as soon as you zypper dup, you will need to again do the reversion.
Comment 3 Pinkernell 2024-02-10 08:55:13 UTC
Thanks Larry. Replacing actual TW's glibc 2.39 by 2.38 from slowroll helped.
Comment 4 Larry Finger 2024-02-10 17:29:37 UTC
For clarity, use the YaST GUI to backport. There are a number of packages that also need to be backported. glibc 2.38 by itself will lock up your system.
Comment 5 Bauglir 2024-02-11 17:09:29 UTC
It's great to have a work-around, but those of you who make use of it should take into account that glibc 2.38 is affected by CVE 2023-6246, rated high (although it requires local access), that was fixed in 2.39.

https://nvd.nist.gov/vuln/detail/CVE-2023-6246
Comment 6 Larry Finger 2024-02-11 17:38:17 UTC
If you control who has physical access to your system, then that CVE is of little concern. Not having access to USB devices from virtual machines is a bigger problem.

Who should be notified about the glibc bug?
Comment 7 Jiri Slaby 2024-02-12 06:12:56 UTC
(In reply to Larry Finger from comment #6)
> Who should be notified about the glibc bug?

Andreas?
Comment 8 Andreas Schwab 2024-02-12 09:36:07 UTC
Please provide a self-contained test case.
Comment 9 Larry Finger 2024-02-12 11:35:03 UTC
Created attachment 872655 [details]
Virtual Machine

This file contains a working virtual machine of openWRT x86_64.
Comment 10 Larry Finger 2024-02-12 12:40:33 UTC
Created attachment 872658 [details]
Trial VirtualBox Machine

This tar file has a minimal VM.
Comment 11 Larry Finger 2024-02-12 12:51:13 UTC
To duplicate this problem, install virtualbox 7.0.14 on a fully updated Tumbleweed host.

Download and unpack the Virtual Machine attachment file. It will show in directory home/finger/Test relative to your currrent working directory. In the VB GUI, select Machine/Add and navigate to home/finger/Test. Select "test.vbox". This step will provide a minimal VM. If you boot it, it will show that it has no bootable medium.

In this machine's screen, pull down the Devices tab and select USB. In the screen that pops up, the options are grayed out. That is the problem.
Comment 12 Ingo Göppert 2024-02-12 13:10:21 UTC
Created attachment 872660 [details]
vboxdrv kernel crash

While starting a VM the vboxdrv kernel module crashes. Log is from a system with tumbleweed from today with a Win 10 VM. Same result with a Win 7 VM.

A crash was also reported here https://www.virtualbox.org/ticket/21964. Both start with "memobj-r0drv-linux.c:564 rtR0MemObjLinuxApplyPageRange+0x4e/0xa0 [vboxdrv]", thus I think it is already reported upstream, but it was ignored there.
Comment 13 Larry Finger 2024-02-12 13:14:32 UTC
I am aware of that WARNING. It does not crash your system. I have not been able to fix the warning, and upstream has not provided a fix as you note.

For the moment, ignore this splat.
Comment 14 Ingo Göppert 2024-02-12 13:19:51 UTC
(In reply to Larry Finger from comment #11)
> To duplicate this problem, install virtualbox 7.0.14 on a fully updated
> Tumbleweed host.
> 
> Download and unpack the Virtual Machine attachment file. It will show in
> directory home/finger/Test relative to your currrent working directory. In
> the VB GUI, select Machine/Add and navigate to home/finger/Test. Select
> "test.vbox". This step will provide a minimal VM. If you boot it, it will
> show that it has no bootable medium.
> 
> In this machine's screen, pull down the Devices tab and select USB. In the
> screen that pops up, the options are grayed out. That is the problem.

Same vboxdrv crash when starting the provided test VMN, but after that virtualbox  created a segfault. I had to 
remove the snapshot to start up the test VM successful.
Comment 15 Ingo Göppert 2024-02-12 13:22:37 UTC
(In reply to Ingo Göppert from comment #14)
> (In reply to Larry Finger from comment #11)
> > To duplicate this problem, install virtualbox 7.0.14 on a fully updated
> > Tumbleweed host.
> > 
> > Download and unpack the Virtual Machine attachment file. It will show in
> > directory home/finger/Test relative to your currrent working directory. In
> > the VB GUI, select Machine/Add and navigate to home/finger/Test. Select
> > "test.vbox". This step will provide a minimal VM. If you boot it, it will
> > show that it has no bootable medium.
> > 
> > In this machine's screen, pull down the Devices tab and select USB. In the
> > screen that pops up, the options are grayed out. That is the problem.
> 
> Same vboxdrv crash when starting the provided test VMN, but after that
> virtualbox  created a segfault. I had to 
> remove the snapshot to start up the test VM successful.

To be clear: successful but with missing USB passthru
Comment 16 Larry Finger 2024-02-12 14:14:54 UTC
(In reply to Ingo Göppert from comment #15)
> 
> To be clear: successful but with missing USB passthru

Exactly.
Comment 17 Javier de San Pedro 2024-02-13 00:12:24 UTC
I am not sure if it is the same issue (though I started facing the same the symptoms during the glibc 2.39 migration), but could you try the following ? 

 chmod 0660 /dev/vboxusb/*/* 

From what I gather from the VBoxCreateUSBNode.sh script, it is supposedly calling mknod with -m 0660 , so no idea why on my system I reproduceably find that these nodes are created with 0640 instead. Anyway, I don't fully understand how this could be even remotely caused by glibc.
Comment 18 Pinkernell 2024-02-13 07:09:10 UTC
(In reply to Javier de San Pedro from comment #17)

>  chmod 0660 /dev/vboxusb/*/* 

Zypper dupped to actual TW, esp. to glibc 2.39-2.1. and 7.0.14-Virt.1699.733.5.

USB is back.
Comment 19 Pinkernell 2024-02-13 07:10:33 UTC
To clarify: I did chmod 0660 /dev/vboxusb/*/* as Javier suggested.
Comment 20 Javier de San Pedro 2024-02-13 23:44:42 UTC
Thanks for confirming. I have looked a bit into it. 

I cannot believe this, but the issue is indeed related to glibc 2.39.

glibc 2.39 has started to use the fchmodat2 syscall, which is used by mknod in the VBoxCreateUSBNode.sh to set the 0660 mode on the vboxusb nodes. 

However, this script is called by udevd, and the systemd unit file for udev has SystemCallFilter= enabled... but the systemd version that we have (v254) is too old to understand the fchmodat2 syscall.

Therefore, mknod succeeds in creating the device node, but fails in chmod 0660'ing it, so it keeps the default 0640 due to the umask. 

Solution is to upgrade to systemd v255 or backport this commit https://github.com/systemd/systemd/commit/6e10405aa25fe5e76b740d9ec59730e3f4470c7a 


<rant please_ignore="yes">
Yet another bug caused by an overzealous system call whitelist. As I said 3 years ago, these whitelists are my number #1 source of bugs on rolling distros. 
https://bugzilla.opensuse.org/show_bug.cgi?id=1190842#c8 

I don't even know why we would even bother with a system call filter on udevd. It is running arbitrary shell scripts and by construction it must have kernel module loading capabilities and whatnot! What threats could a syscall filter for udevd possibly protect against ? Literally the only thing it does is complicate debugging (no ptrace!) and create issues like this. </rant>
Comment 21 Larry Finger 2024-02-14 17:07:30 UTC
Thanks for chasing this down. Although it was a bit uncomfortable being the "canary in the coal mine," we are happy to contribute. :)


I will be watching for an update to glibc.
Comment 22 robert spitzenpfeil 2024-02-14 21:36:08 UTC
Adding

chmod 0660 "$devpath" 2>/dev/null

after the mknod line in said script helps for now.
Comment 23 Thomas Rother 2024-02-18 21:24:28 UTC
The current version
ID="opensuse-tumbleweed"
ID_LIKE="opensuse suse"
VERSION_ID="20240216"

together with chmod 0660 /dev/vboxusb/*/*  solved the USB issues with vbox 7.0.14_SUSE r161095 in my case. I am happy that I can throw vmware-workstation into the waste bin now ;-)
Comment 24 Pinkernell 2024-02-19 07:18:02 UTC
chmod 0660 still needs to be applied by sudo every time a USB device is (re)connected. I do not believe that this is expected behaviour. So is this really resolved and fixed? Has anyone tried with systemd v255? I do not know where to get hold of it.
Comment 25 Thomas Rother 2024-02-19 07:21:52 UTC
switching  back to "reopened". I also have no chance to test this finally ...
Comment 26 Jiri Slaby 2024-02-19 08:21:37 UTC
systemd 255 is not even in a devel project, so how this came as RESOLVED?
Comment 28 Franck Bui 2024-02-22 11:11:08 UTC
(In reply to Javier de San Pedro from comment #20)
> Solution is to upgrade to systemd v255 or backport this commit
> https://github.com/systemd/systemd/commit/
> 6e10405aa25fe5e76b740d9ec59730e3f4470c7a 

Thanks for looking into this.

This commit has been backported and submitted (sr#1149219).

Sorry if it took so long.
Comment 29 OBSbugzilla Bot 2024-02-22 11:35:04 UTC
This is an autogenerated message for OBS integration:
This bug (1219766) was mentioned in
https://build.opensuse.org/request/show/1149219 Factory / systemd
Comment 30 Franck Bui 2024-02-22 15:57:33 UTC
The fix should be part of TW soon (most likely next week if nothing wrong pops up), so I'll close the bug.

Thanks all for your help/feedback and again sorry for the delay.
Comment 32 Larry Finger 2024-02-24 18:52:57 UTC
I have verified that systemd 254 in Tumbleweed snapshot 20240223 fixes the problem with USB passthrough in VirtualBox 7.0.14.

Thanks for the effort is getting the new version of systemd into TW.
Comment 33 Mario Guzman 2024-02-24 19:01:13 UTC
Same here, AOK now.
Comment 34 T Adams 2024-02-25 06:53:24 UTC
The 20240223 snapshot also resolved the passthrough-to-USB-scanners issue described at: https://forums.opensuse.org/t/usb-passthrough-to-scanner-again-dead-in-tumbleweed-works-in-leap-15-5/172292.  Thanks to everyone who contributed to fixing this problem.
Comment 35 OBSbugzilla Bot 2024-02-27 11:35:06 UTC
This is an autogenerated message for OBS integration:
This bug (1219766) was mentioned in
https://build.opensuse.org/request/show/1152118 Factory / systemd
Comment 37 Thomas Rother 2024-04-01 13:03:01 UTC
... it seems like this strange bug reappeared again. The USB devices are gone on all vbox guests, even with the starting user being a member of "vboxusers" And the workaround from https://bugzilla.opensuse.org/show_bug.cgi?id=1219766#c17 with

chmod 0660 /dev/vboxusb/*/*

does not solve this anymore. Current versions are:

NAME="openSUSE Tumbleweed"
# VERSION="20240329"
ID="opensuse-tumbleweed"
ID_LIKE="opensuse suse"
VERSION_ID="20240329"
PRETTY_NAME="openSUSE Tumbleweed"


Other version infos:

information for package virtualbox:
-----------------------------------
Repository     : repo-oss
Name           : virtualbox
Version        : 7.0.14-6.1
Arch           : x86_64
Vendor         : openSUSE

Information for package glibc:
------------------------------
Repository     : repo-oss
Name           : glibc
Version        : 2.39-4.2
Arch           : x86_64
Vendor         : openSUSE

information for package udev:
-----------------------------
Repository     : repo-oss
Name           : udev
Version        : 255.4-1.3
Arch           : x86_64
Vendor         : openSUSE


I am open for any ideas to debug this ;-( ...
Comment 38 Javier de San Pedro 2024-04-01 13:04:27 UTC
It is not the same issue. This new issue is being discussed in bug 1222186 .
Comment 39 Thomas Rother 2024-04-01 13:53:33 UTC
Note: new issues with USB passthrough are discussed at https://bugzilla.suse.com/show_bug.cgi?id=1219766
Comment 40 Larry Finger 2024-04-05 23:04:05 UTC
(In reply to Thomas Rother from comment #37)
> ... it seems like this strange bug reappeared again. The USB devices are
> gone on all vbox guests, even with the starting user being a member of
> "vboxusers" And the workaround from
> https://bugzilla.opensuse.org/show_bug.cgi?id=1219766#c17 with
> 
> chmod 0660 /dev/vboxusb/*/*

What permissions do you have on /dev/vboxusb? It shoudl be 0750 as in 

~>ls -l /dev/vboxusb
drwxr-x--- 2 root vboxusers 100 Apr  5 17:20
Comment 41 Thomas Rother 2024-04-06 06:33:32 UTC
(In reply to Larry Finger from comment #40)
> (In reply to Thomas Rother from comment #37)
> > ... it seems like this strange bug reappeared again. The USB devices are
> > gone on all vbox guests, even with the starting user being a member of
> > "vboxusers" And the workaround from
> > https://bugzilla.opensuse.org/show_bug.cgi?id=1219766#c17 with
> > 
> > chmod 0660 /dev/vboxusb/*/*
> 
> What permissions do you have on /dev/vboxusb? It shoudl be 0750 as in 
> 
> ~>ls -l /dev/vboxusb
> drwxr-x--- 2 root vboxusers 100 Apr  5 17:20

I have the same

thommie@odysseus4:~> ls -l /dev/vboxusb
insgesamt 0
drwxr-x--- 2 root vboxusers 180  6. Apr 08:28 001
drwxr-x--- 2 root vboxusers  60  6. Apr 08:22 004

The issue was solved with https://bugzilla.suse.com/show_bug.cgi?id=1222186 and is still OK on my TB 20240403