Bug 1219216 - Bluetooth: Race condition between MGMT_OP_REMOVE_ADV_MONITOR command with power of controller
Summary: Bluetooth: Race condition between MGMT_OP_REMOVE_ADV_MONITOR command with pow...
Status: RESOLVED FIXED
Alias: None
Product: PUBLIC SUSE Linux Enterprise Server 15 SP6
Classification: openSUSE
Component: Other (show other bugs)
Version: unspecified
Hardware: aarch64 SLES 15
: P2 - High : Normal
Target Milestone: ---
Assignee: Joey Lee
QA Contact:
URL: https://openqa.suse.de/tests/13350211...
Whiteboard: in integration
Keywords:
Depends on:
Blocks:
 
Reported: 2024-01-26 09:49 UTC by Zaoliang Luo
Modified: 2024-07-08 14:31 UTC (History)
11 users (show)

See Also:
Found By: openQA
Services Priority:
Business Priority:
Blocker: Yes
Marketing QA Status: ---
IT Deployment: ---


Attachments
yast logs (474.04 KB, application/x-xz)
2024-01-26 09:51 UTC, Zaoliang Luo
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Zaoliang Luo 2024-01-26 09:49:07 UTC
I have checked this manually on my RPi4 model b. 
I found following issue:


4mb:~ # systemctl start bluetooth
4mb:~ # systemctl status bluetooth
● bluetooth.service - Bluetooth service
     Loaded: loaded (/usr/lib/systemd/system/bluetooth.service; enabled; preset: disabled)
     Active: active (running) since Fri 2024-01-26 09:45:14 UTC; 1s ago
       Docs: man:bluetoothd(8)
   Main PID: 20976 (bluetoothd)
     Status: "Running"
      Tasks: 1
        CPU: 119ms
     CGroup: /system.slice/bluetooth.service
             └─20976 /usr/lib/bluetooth/bluetoothd

Jan 26 09:45:14 4mb bluetoothd[20976]: src/plugin.c:plugin_init() System does not support admin plugin
Jan 26 09:45:14 4mb bluetoothd[20976]: Bluetooth management interface 1.22 initialized
Jan 26 09:45:14 4mb bluetoothd[20976]: src/adapter.c:reset_adv_monitors_complete() Failed to reset Adv Monitors: Failed>
Jan 26 09:45:14 4mb bluetoothd[20976]: Failed to clear UUIDs: Failed (0x03)
Jan 26 09:45:14 4mb bluetoothd[20976]: Failed to add UUID: Failed (0x03)
Jan 26 09:45:14 4mb bluetoothd[20976]: Failed to add UUID: Failed (0x03)
Jan 26 09:45:14 4mb bluetoothd[20976]: Failed to add UUID: Failed (0x03)
Jan 26 09:45:14 4mb bluetoothd[20976]: Failed to add UUID: Failed (0x03)
Jan 26 09:45:14 4mb bluetoothd[20976]: Failed to add UUID: Failed (0x03)
Jan 26 09:45:14 4mb bluetoothd[20976]: Failed to add UUID: Failed (0x03) 



## Observation

openQA test in scenario sle-15-SP6-JeOS-for-RaspberryPi-aarch64-jeos-realhw-RPi@RPi4 fails in
[rpi_bluetooth](https://openqa.suse.de/tests/13350211/modules/rpi_bluetooth/steps/9)

## Test suite description
Run a test on real RPi hardware. To be used eg. with RPi4 machine.
Setup: https://confluence.suse.com/pages/viewpage.action?spaceKey=~dheidler&title=Hardware+Automation


## Reproducible

Fails since (at least) Build [2.5](https://openqa.suse.de/tests/12506382)


## Expected result

Last good: (unknown) (or more recent)


## Further details

Always latest result in this scenario: [latest](https://openqa.suse.de/tests/latest?arch=aarch64&distri=sle&flavor=JeOS-for-RaspberryPi&machine=RPi4&test=jeos-realhw-RPi&version=15-SP6)
Comment 1 Zaoliang Luo 2024-01-26 09:51:54 UTC
Created attachment 872219 [details]
yast logs
Comment 2 Swayammitra Tripathy 2024-02-19 10:21:38 UTC
Any Progress on this bug or still it is a blocker?
Comment 3 Joey Lee 2024-02-19 14:05:41 UTC
(In reply to Zaoliang Luo from comment #0)
> I have checked this manually on my RPi4 model b. 
> I found following issue:
> 
> 
> 4mb:~ # systemctl start bluetooth
> 4mb:~ # systemctl status bluetooth
> ● bluetooth.service - Bluetooth service
>      Loaded: loaded (/usr/lib/systemd/system/bluetooth.service; enabled;
> preset: disabled)
>      Active: active (running) since Fri 2024-01-26 09:45:14 UTC; 1s ago
>        Docs: man:bluetoothd(8)
>    Main PID: 20976 (bluetoothd)
>      Status: "Running"
>       Tasks: 1
>         CPU: 119ms
>      CGroup: /system.slice/bluetooth.service
>              └─20976 /usr/lib/bluetooth/bluetoothd
> 
> Jan 26 09:45:14 4mb bluetoothd[20976]: src/plugin.c:plugin_init() System
> does not support admin plugin
> Jan 26 09:45:14 4mb bluetoothd[20976]: Bluetooth management interface 1.22
> initialized
> Jan 26 09:45:14 4mb bluetoothd[20976]:
> src/adapter.c:reset_adv_monitors_complete() Failed to reset Adv Monitors:
> Failed>
> Jan 26 09:45:14 4mb bluetoothd[20976]: Failed to clear UUIDs: Failed (0x03)
> Jan 26 09:45:14 4mb bluetoothd[20976]: Failed to add UUID: Failed (0x03)
> Jan 26 09:45:14 4mb bluetoothd[20976]: Failed to add UUID: Failed (0x03)
> Jan 26 09:45:14 4mb bluetoothd[20976]: Failed to add UUID: Failed (0x03)
> Jan 26 09:45:14 4mb bluetoothd[20976]: Failed to add UUID: Failed (0x03)
> Jan 26 09:45:14 4mb bluetoothd[20976]: Failed to add UUID: Failed (0x03)
> Jan 26 09:45:14 4mb bluetoothd[20976]: Failed to add UUID: Failed (0x03) 
> 

Looks that it's similar with this issue:

https://bugs.archlinux.org/task/78980
https://bugzilla.kernel.org/show_bug.cgi?id=217651

The kernel patch is:

commit 95b7015433053cd5f648ad2a7b8f43b2c99c949a         [v6.5-rc3~20^2^2~1]
Author: Tomasz Moń <tomasz.mon@nordicsemi.no>
Date:   Thu Jul 13 12:25:14 2023 +0200

    Bluetooth: btusb: Fix bluetooth on Intel Macbook 2014

    Commit c13380a55522 ("Bluetooth: btusb: Do not require hardcoded
    interface numbers") inadvertedly broke bluetooth on Intel Macbook 2014.
    The intention was to keep behavior intact when BTUSB_IFNUM_2 is set and
    otherwise allow any interface numbers. The problem is that the new logic
    condition omits the case where bInterfaceNumber is 0.

    Fix BTUSB_IFNUM_2 handling by allowing both interface number 0 and 2
    when the flag is set.

    Fixes: c13380a55522 ("Bluetooth: btusb: Do not require hardcoded interface numbers")    [v6.4-rc1~77^2~46^2~70]
    Reported-by: John Holland <johnbholland@icloud.com>
    Closes: https://bugzilla.kernel.org/show_bug.cgi?id=217651
    Signed-off-by: Tomasz Moń <tomasz.mon@nordicsemi.no>
    Tested-by: John Holland<johnbholland@icloud.com>
    Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>

But this patch is already merged to 15-SP6 kernel:

patches.suse/Bluetooth-btusb-Fix-bluetooth-on-Intel-Macbook-.patch              [SLE-15-SP6-Full-Alpha-202310-2~2502]

Hi Zaoliang, 

Did you update your 15-SP6 kernel to the latest version?

Thanks!
Comment 4 Joey Lee 2024-02-19 14:24:51 UTC
(In reply to Joey Lee from comment #3)
> (In reply to Zaoliang Luo from comment #0)
[..snip]
> But this patch is already merged to 15-SP6 kernel:
> 
> patches.suse/Bluetooth-btusb-Fix-bluetooth-on-Intel-Macbook-.patch          
> [SLE-15-SP6-Full-Alpha-202310-2~2502]
> 
> Hi Zaoliang, 
> 
> Did you update your 15-SP6 kernel to the latest version?
> 

And, also update bluez to v5.70 which alignment with 15-SP6 IBS repo, please.
Comment 5 Zaoliang Luo 2024-02-19 15:21:34 UTC
(In reply to Joey Lee from comment #4)
> (In reply to Joey Lee from comment #3)
> > (In reply to Zaoliang Luo from comment #0)
> [..snip]
> > But this patch is already merged to 15-SP6 kernel:
> > 
> > patches.suse/Bluetooth-btusb-Fix-bluetooth-on-Intel-Macbook-.patch          
> > [SLE-15-SP6-Full-Alpha-202310-2~2502]
> > 
> > Hi Zaoliang, 
> > 
> > Did you update your 15-SP6 kernel to the latest version?
> > 
> 
> And, also update bluez to v5.70 which alignment with 15-SP6 IBS repo, please.

Please take a look at https://openqa.suse.de/tests/13493430#step/rpi_bluetooth/9

this is still failing on openQA:

sle-15-SP6-JeOS-for-RaspberryPi-aarch64-Build4.62


I am not very sure whether bluez version is updated or not. We use aarch64-Build54.1 on openQA.

I can do a manual installation for further check. please provide the link for updated bluez package from IBS repo, thanks!
Comment 6 Zaoliang Luo 2024-02-19 16:22:49 UTC
I installed latest build and I can still reproduce this issue:

4mb:~ # zypper in bluez
Refreshing service 'Basesystem_Module_15_SP6_aarch64'.
Refreshing service 'Containers_Module_15_SP6_aarch64'.
Refreshing service 'Desktop_Applications_Module_15_SP6_aarch64'.
Refreshing service 'Development_Tools_Module_15_SP6_aarch64'.
Refreshing service 'Legacy_Module_15_SP6_aarch64'.
Refreshing service 'Public_Cloud_Module_15_SP6_aarch64'.
Refreshing service 'Python_3_Module_15_SP6_aarch64'.
Refreshing service 'SUSE_Linux_Enterprise_Server_15_SP6_aarch64'.
Refreshing service 'SUSE_Package_Hub_15_SP6_aarch64'.
Refreshing service 'Server_Applications_Module_15_SP6_aarch64'.
Refreshing service 'Transactional_Server_Module_15_SP6_aarch64'.
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following 2 NEW packages are going to be installed:
  bluez libasound2

2 new packages to install.
Overall download size: 1.8 MiB. Already cached: 0 B. After the operation, additional 8.0 MiB will be used.
Continue? [y/n/v/...? shows all options] (y): y
Retrieving: libasound2-1.2.10-150600.2.2.aarch64 (SLE-Module-Basesystem15-SP6-Pool)                        (1/2), 430.2 KiB    
Retrieving: libasound2-1.2.10-150600.2.2.aarch64.rpm ....................................................................[done]
Retrieving: bluez-5.70-150600.1.2.aarch64 (SLE-Module-Basesystem15-SP6-Pool)                               (2/2),   1.4 MiB    
Retrieving: bluez-5.70-150600.1.2.aarch64.rpm .............................................................[done (704.0 KiB/s)]

Checking for file conflicts: ............................................................................................[done]
(1/2) Installing: libasound2-1.2.10-150600.2.2.aarch64 ..................................................................[done]
Created symlink /etc/systemd/system/dbus-org.bluez.service -> /usr/lib/systemd/system/bluetooth.service.
Created symlink /etc/systemd/system/bluetooth.target.wants/bluetooth.service -> /usr/lib/systemd/system/bluetooth.service.
(2/2) Installing: bluez-5.70-150600.1.2.aarch64 .........................................................................[done]
Running post-transaction scripts ........................................................................................[done]
4mb:~ # systemctl status bluetooth
○ bluetooth.service - Bluetooth service
     Loaded: loaded (/usr/lib/systemd/system/bluetooth.service; enabled; preset: disabled)
     Active: inactive (dead)
       Docs: man:bluetoothd(8)
4mb:~ # systemctl start bluetooth
4mb:~ # systemctl status bluetooth
● bluetooth.service - Bluetooth service
     Loaded: loaded (/usr/lib/systemd/system/bluetooth.service; enabled; preset: disabled)
     Active: active (running) since Mon 2024-02-19 16:18:37 UTC; 4s ago
       Docs: man:bluetoothd(8)
   Main PID: 16061 (bluetoothd)
     Status: "Running"
      Tasks: 1
        CPU: 115ms
     CGroup: /system.slice/bluetooth.service
             └─16061 /usr/lib/bluetooth/bluetoothd

Feb 19 16:18:37 4mb bluetoothd[16061]: src/plugin.c:plugin_init() System does not support admin plugin
Feb 19 16:18:37 4mb bluetoothd[16061]: Bluetooth management interface 1.22 initialized
Feb 19 16:18:37 4mb bluetoothd[16061]: src/adapter.c:reset_adv_monitors_complete() Failed to reset Adv Monitors: Failed (0x03)
Feb 19 16:18:37 4mb bluetoothd[16061]: Failed to clear UUIDs: Failed (0x03)
Feb 19 16:18:37 4mb bluetoothd[16061]: Failed to add UUID: Failed (0x03)
Feb 19 16:18:37 4mb bluetoothd[16061]: Failed to add UUID: Failed (0x03)
Feb 19 16:18:37 4mb bluetoothd[16061]: Failed to add UUID: Failed (0x03)
Feb 19 16:18:37 4mb bluetoothd[16061]: Failed to add UUID: Failed (0x03)
Feb 19 16:18:37 4mb bluetoothd[16061]: Failed to add UUID: Failed (0x03)
Feb 19 16:18:37 4mb bluetoothd[16061]: Failed to add UUID: Failed (0x03)

---

bluez-5.70-150600.1.2.aarch64 got installed on my RPi 4 model B.


4mb:~ # cat /etc/os-release
NAME="SLES"
VERSION="15-SP6"
VERSION_ID="15.6"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP6"
ID="sles"
ID_LIKE="suse"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:15:sp6"
DOCUMENTATION_URL="https://documentation.suse.com/"
4mb:~ # uname -a
Linux 4mb 6.4.0-150600.7-default #1 SMP PREEMPT_DYNAMIC Thu Feb  8 20:18:21 UTC 2024 (eafe2c2) aarch64 aarch64 aarch64 GNU/Linux
Comment 7 Joey Lee 2024-02-22 03:48:21 UTC
Hi 

(In reply to Zaoliang Luo from comment #5)
> (In reply to Joey Lee from comment #4)
> > (In reply to Joey Lee from comment #3)
> > > (In reply to Zaoliang Luo from comment #0)
> > [..snip]
> > > But this patch is already merged to 15-SP6 kernel:
> > > 
> > > patches.suse/Bluetooth-btusb-Fix-bluetooth-on-Intel-Macbook-.patch          
> > > [SLE-15-SP6-Full-Alpha-202310-2~2502]
> > > 
> > > Hi Zaoliang, 
> > > 
> > > Did you update your 15-SP6 kernel to the latest version?
> > > 
> > 
> > And, also update bluez to v5.70 which alignment with 15-SP6 IBS repo, please.
> 
> Please take a look at
> https://openqa.suse.de/tests/13493430#step/rpi_bluetooth/9
> 
> this is still failing on openQA:
> 
> sle-15-SP6-JeOS-for-RaspberryPi-aarch64-Build4.62
> 

Where can I download the above image for RaspberryPi 4?

I found a Raspberry Pi 4 but I didn't find the above image from the following release URL:

Download Links
==============
You can get the Snapshot-202402-1 ISO images from:

- [SLE-15-SP6-Full-Snapshot-202402-1](
http://download.suse.de/install/SLE-15-SP6-Full-Snapshot-202402-1)
- [SLE-15-SP6-Online-Snapshot-202402-1](
http://download.suse.de/install/SLE-15-SP6-Online-Snapshot-202402-1)
- [SLE-15-SP6-Minimal-Snapshot-202402-1](
http://download.suse.de/install/SLE-15-SP6-Minimal-Snapshot-202402-1)

Could you please share the link for downloading sle-15-SP6-JeOS-for-RaspberryPi-aarch64-Build4.62 image?

Thanks a lot!
Comment 8 Zaoliang Luo 2024-02-22 07:25:03 UTC
(In reply to Joey Lee from comment #7)
> Hi 
> 
> (In reply to Zaoliang Luo from comment #5)
> > (In reply to Joey Lee from comment #4)
> > > (In reply to Joey Lee from comment #3)
> > > > (In reply to Zaoliang Luo from comment #0)
> > > [..snip]
> > > > But this patch is already merged to 15-SP6 kernel:
> > > > 
> > > > patches.suse/Bluetooth-btusb-Fix-bluetooth-on-Intel-Macbook-.patch          
> > > > [SLE-15-SP6-Full-Alpha-202310-2~2502]
> > > > 
> > > > Hi Zaoliang, 
> > > > 
> > > > Did you update your 15-SP6 kernel to the latest version?
> > > > 
> > > 
> > > And, also update bluez to v5.70 which alignment with 15-SP6 IBS repo, please.
> > 
> > Please take a look at
> > https://openqa.suse.de/tests/13493430#step/rpi_bluetooth/9
> > 
> > this is still failing on openQA:
> > 
> > sle-15-SP6-JeOS-for-RaspberryPi-aarch64-Build4.62
> > 
> 
> Where can I download the above image for RaspberryPi 4?
> 
> I found a Raspberry Pi 4 but I didn't find the above image from the
> following release URL:
> 
> Download Links
> ==============
> You can get the Snapshot-202402-1 ISO images from:
> 
> - [SLE-15-SP6-Full-Snapshot-202402-1](
> http://download.suse.de/install/SLE-15-SP6-Full-Snapshot-202402-1)
> - [SLE-15-SP6-Online-Snapshot-202402-1](
> http://download.suse.de/install/SLE-15-SP6-Online-Snapshot-202402-1)
> - [SLE-15-SP6-Minimal-Snapshot-202402-1](
> http://download.suse.de/install/SLE-15-SP6-Minimal-Snapshot-202402-1)
> 
> Could you please share the link for downloading
> sle-15-SP6-JeOS-for-RaspberryPi-aarch64-Build4.62 image?
> 
> Thanks a lot!

actually you can download qcow2 directly: https://openqa.suse.de/tests/13493430#downloads

or you can get raw always from here:

https://download.suse.de/ibs/SUSE:/SLE-15-SP6:/GA:/TEST/images/SLES15-SP6-Minimal-Image.aarch64-RaspberryPi-Build4.88.raw.xz
Comment 9 Joey Lee 2024-02-23 09:39:02 UTC
(In reply to Zaoliang Luo from comment #8)
> (In reply to Joey Lee from comment #7)
[...snip]
> 
> actually you can download qcow2 directly:
> https://openqa.suse.de/tests/13493430#downloads
> 
> or you can get raw always from here:
> 
> https://download.suse.de/ibs/SUSE:/SLE-15-SP6:/GA:/TEST/images/SLES15-SP6-
> Minimal-Image.aarch64-RaspberryPi-Build4.88.raw.xz

I have installed the above image and the error log can be reproduced on my Raspberry pi 4.
I think that the reason is _Minimal_ image is too mini. It doesn't include kernel-firmware-bluetooth RPM. 

After installed kernel-firmware-bluetooth-20240201-150600.1.1.noarch.rpm, the bluetoothd can be enabled on my Raspberrypi 4. And bluetoothctl tool works for power-on host for scanning devices.

Hi Zaoliang, 

Could you please help to check kernel-firmware-bluetooth? If you did not install it, please install and reboot.

On the other hand, looks that the _Minimal Image_ should NOT run bluetooth test cases. Unless we add kernel-firmware-bluetooth to the image.
Comment 10 Zaoliang Luo 2024-02-23 13:13:38 UTC
(In reply to Joey Lee from comment #9)
> (In reply to Zaoliang Luo from comment #8)
> > (In reply to Joey Lee from comment #7)
> [...snip]
> > 
> > actually you can download qcow2 directly:
> > https://openqa.suse.de/tests/13493430#downloads
> > 
> > or you can get raw always from here:
> > 
> > https://download.suse.de/ibs/SUSE:/SLE-15-SP6:/GA:/TEST/images/SLES15-SP6-
> > Minimal-Image.aarch64-RaspberryPi-Build4.88.raw.xz
> 
> I have installed the above image and the error log can be reproduced on my
> Raspberry pi 4.
> I think that the reason is _Minimal_ image is too mini. It doesn't include
> kernel-firmware-bluetooth RPM. 
> 
> After installed kernel-firmware-bluetooth-20240201-150600.1.1.noarch.rpm,
> the bluetoothd can be enabled on my Raspberrypi 4. And bluetoothctl tool
> works for power-on host for scanning devices.
> 
> Hi Zaoliang, 
> 
> Could you please help to check kernel-firmware-bluetooth? If you did not
> install it, please install and reboot.
> 
> On the other hand, looks that the _Minimal Image_ should NOT run bluetooth
> test cases. Unless we add kernel-firmware-bluetooth to the image.

I tested build4.66 jeos on my rpi 4 model b.
I also checked on openQA, this bluetooth is working now. 

I want clarify here. We do this test on openQA for jeos and we can detect this kind of issue in time. this is why we do for basic stuff.

The failed messages about UUID doesn't seem to be directly related to broken function of bluetooth itself because this is still there on openQA:

https://openqa.suse.de/tests/13548531#step/rpi_bluetooth/9

On jeos we can run registration to be able get repositories which can be used for further testing.

This is the way I'm doing manually. Without installing kernel-firmware-bluetooth 20240115-150600.1.1 it is working already with bluetooth.

After I installed kernel-firmware-bluetooth and rebooted rpi, I checked bluetooth:


localhost:~ # systemctl status bluetooth
● bluetooth.service - Bluetooth service
     Loaded: loaded (/usr/lib/systemd/system/bluetooth.service; enabled; preset: disabled)
     Active: active (running) since Wed 2024-02-14 12:00:13 UTC; 1 week 2 days ago
       Docs: man:bluetoothd(8)
   Main PID: 726 (bluetoothd)
     Status: "Running"
      Tasks: 1
        CPU: 151ms
     CGroup: /system.slice/bluetooth.service
             └─726 /usr/lib/bluetooth/bluetoothd

Feb 14 12:00:13 localhost bluetoothd[726]: Starting SDP server
Feb 14 12:00:13 localhost bluetoothd[726]: src/plugin.c:plugin_init() System does not support csip plugin
Feb 14 12:00:13 localhost bluetoothd[726]: profiles/audio/micp.c:micp_init() D-Bus experimental not enabled
Feb 14 12:00:13 localhost bluetoothd[726]: src/plugin.c:plugin_init() System does not support micp plugin
Feb 14 12:00:13 localhost bluetoothd[726]: src/plugin.c:plugin_init() System does not support vcp plugin
Feb 14 12:00:13 localhost bluetoothd[726]: src/plugin.c:plugin_init() System does not support mcp plugin
Feb 14 12:00:13 localhost bluetoothd[726]: src/plugin.c:plugin_init() System does not support bass plugin
Feb 14 12:00:13 localhost bluetoothd[726]: src/plugin.c:plugin_init() System does not support bap plugin
Feb 14 12:00:13 localhost bluetoothd[726]: src/plugin.c:plugin_init() System does not support admin plugin
Feb 14 12:00:13 localhost bluetoothd[726]: Bluetooth management interface 1.22 initialized


--
This looks better and I can confirm that this issue has been resolved. 
For customer it is better to have already working bluetooth, not facing this kind of issue. It means I would suggest to include bluetooth firmware for jeos because it should be supported, thanks!
Comment 11 Joey Lee 2024-02-24 08:00:17 UTC
Set this issue to FIXED.
Comment 12 Zaoliang Luo 2024-04-15 07:48:12 UTC
(In reply to Joey Lee from comment #11)
> Set this issue to FIXED.

Sorry, kernel-firmware-bluetooth is not pre-installed in minimal image and provided, we have this issue not solved.

The user has no idea what is needed to fix this problem. 

We still have the same issue on openQA, please fix it.
Comment 13 Joey Lee 2024-04-15 08:10:47 UTC
(In reply to Zaoliang Luo from comment #12)
> (In reply to Joey Lee from comment #11)
> > Set this issue to FIXED.
> 
> Sorry, kernel-firmware-bluetooth is not pre-installed in minimal image and
> provided, we have this issue not solved.
> 
> The user has no idea what is needed to fix this problem. 
> 
> We still have the same issue on openQA, please fix it.

Could you please check with guy who handles minimal image? It's not bluetooth problem. It's a image installation problem.
Comment 14 Joey Lee 2024-04-15 08:13:44 UTC
Hi Andreas,

Sorry for bother you!

Do you know that who handles the installation of ARM64 minimal image? Do we support bluetooth in ARM64 minimal image by default? If so, then someone should add kernel-firmware-bluetooth package to ARM64 minimal image.

Thanks!
Comment 15 Zaoliang Luo 2024-04-15 08:58:42 UTC
I think Andreas can help :)
Comment 16 Fabian Vogt 2024-04-15 13:57:13 UTC
Is kernel-firmware-bluetooth really required? So far the needed firmware files for bluetooth were either in kernel-firmware-brcm or bcm43xx-firmware.
Comment 17 Radoslav Tzvetkov 2024-04-15 14:33:52 UTC
Anreas, can you please address this bug it is set as a blocker for PublicRC. Do you think it can be resolved before the submit deadline of 17.4 ?
Comment 18 OBSbugzilla Bot 2024-04-15 14:35:03 UTC
This is an autogenerated message for OBS integration:
This bug (1219216) was mentioned in
https://build.opensuse.org/request/show/1167822 Factory / kiwi-templates-Minimal
Comment 19 Joey Lee 2024-04-16 08:47:11 UTC
I have re-installed minimal image and test again. I found that it's not about kernel-firmware-bluetooth, it's about when is the bluetooth daemon started. If the bluetooth daemon be started manually by user in terminal, then reset_adv_monitors_complete() failed can be reproduced. 

But if we set enable by "systemctl enable bluetooth" and reboot. Then reset_adv_monitors_complete() failed just gone forever. After I installed kernel-firmware-bluetooth and rebooted, I also set bluetooth daemon enabled. So the failed was gone.

The minimal image does not include bluez. So user needs to install manually. On the other hand, after bluetooth daemon be set to enabled. The reset_adv_monitors_complete() failed is also gone. 

Base on the code path index_added()->reset_adv_monitors() in bluez. Looks that the fail message is a warning which does not affect the initial for controller. I can still power on controller and scan devices success.
Comment 20 Zaoliang Luo 2024-04-16 09:07:26 UTC
(In reply to Joey Lee from comment #19)
> I have re-installed minimal image and test again. I found that it's not
> about kernel-firmware-bluetooth, it's about when is the bluetooth daemon
> started. If the bluetooth daemon be started manually by user in terminal,
> then reset_adv_monitors_complete() failed can be reproduced. 
> 
> But if we set enable by "systemctl enable bluetooth" and reboot. Then
> reset_adv_monitors_complete() failed just gone forever. After I installed
> kernel-firmware-bluetooth and rebooted, I also set bluetooth daemon enabled.
> So the failed was gone.
> 
> The minimal image does not include bluez. So user needs to install manually.
> On the other hand, after bluetooth daemon be set to enabled. The
> reset_adv_monitors_complete() failed is also gone. 
> 
> Base on the code path index_added()->reset_adv_monitors() in bluez. Looks
> that the fail message is a warning which does not affect the initial for
> controller. I can still power on controller and scan devices success.

output on my device:

before 'zypper in kernel-firmware-bluetooth'

 
localhost:~ # systemctl status bluetooth
● bluetooth.service - Bluetooth service
     Loaded: loaded (/usr/lib/systemd/system/bluetooth.service; enabled; preset: disabled)
     Active: active (running) since Mon 2024-04-15 09:39:34 CEST; 55s ago
       Docs: man:bluetoothd(8)
   Main PID: 1536 (bluetoothd)
     Status: "Running"
      Tasks: 1
        CPU: 116ms
     CGroup: /system.slice/bluetooth.service
             └─1536 /usr/lib/bluetooth/bluetoothd

Apr 15 09:39:34 localhost bluetoothd[1536]: Bluetooth management interface 1.22 initialized
Apr 15 09:39:34 localhost bluetoothd[1536]: src/adapter.c:reset_adv_monitors_complete() Failed to reset Adv Monitors: Failed (0x>
Apr 15 09:39:34 localhost bluetoothd[1536]: Battery Provider Manager created
Apr 15 09:39:34 localhost bluetoothd[1536]: Failed to clear UUIDs: Failed (0x03)
Apr 15 09:39:34 localhost bluetoothd[1536]: Failed to add UUID: Failed (0x03)
Apr 15 09:39:34 localhost bluetoothd[1536]: Failed to add UUID: Failed (0x03)
Apr 15 09:39:34 localhost bluetoothd[1536]: Failed to add UUID: Failed (0x03)
Apr 15 09:39:34 localhost bluetoothd[1536]: Failed to add UUID: Failed (0x03)
Apr 15 09:39:34 localhost bluetoothd[1536]: Failed to add UUID: Failed (0x03)
Apr 15 09:39:34 localhost bluetoothd[1536]: Failed to add UUID: Failed (0x03)


---
After installation of kernel-firmware-bluetooth

localhost:~ # reboot
localhost:~ # Connection to 192.168.8.149 closed by remote host.
Mac-mini:Work-Docs Zaoliang$ ssh zaoliang@192.168.8.149
(zaoliang@192.168.8.149) Password: 
Last login: Thu Apr 11 09:10:57 2024 from console
zaoliang@localhost:~> su
Passwort: 
localhost:/home/zaoliang # systemctl status bluetooth
● bluetooth.service - Bluetooth service
     Loaded: loaded (/usr/lib/systemd/system/bluetooth.service; enabled; preset: disabled)
     Active: active (running) since Thu 2024-02-22 13:00:27 CET; 1 month 22 days ago
       Docs: man:bluetoothd(8)
   Main PID: 780 (bluetoothd)
     Status: "Running"
      Tasks: 1
        CPU: 139ms
     CGroup: /system.slice/bluetooth.service
             └─780 /usr/lib/bluetooth/bluetoothd

Feb 22 13:00:27 localhost bluetoothd[780]: src/plugin.c:plugin_init() System does not support csip plugin
Feb 22 13:00:27 localhost bluetoothd[780]: profiles/audio/micp.c:micp_init() D-Bus experimental not enabled
Feb 22 13:00:27 localhost bluetoothd[780]: src/plugin.c:plugin_init() System does not support micp plugin
Feb 22 13:00:27 localhost bluetoothd[780]: src/plugin.c:plugin_init() System does not support vcp plugin
Feb 22 13:00:27 localhost bluetoothd[780]: src/plugin.c:plugin_init() System does not support mcp plugin
Feb 22 13:00:27 localhost bluetoothd[780]: src/plugin.c:plugin_init() System does not support bass plugin
Feb 22 13:00:27 localhost bluetoothd[780]: src/plugin.c:plugin_init() System does not support bap plugin
Feb 22 13:00:27 localhost bluetoothd[780]: src/plugin.c:plugin_init() System does not support admin plugin
Feb 22 13:00:27 localhost bluetoothd[780]: Bluetooth management interface 1.22 initialized
Feb 22 13:00:27 localhost bluetoothd[780]: Battery Provider Manager created


--

So the difference is clear, without kernel-firmware-bluetooth to me, it doesn't work if kernel-firmware-bluetooth is available.
Comment 21 Fabian Vogt 2024-04-16 11:56:48 UTC
(In reply to Fabian Vogt from comment #16)
> Is kernel-firmware-bluetooth really required? So far the needed firmware
> files for bluetooth were either in kernel-firmware-brcm or bcm43xx-firmware.

^

If kernel-firmware-bluetooth is really needed, it's possible to check which file(s) in particular are needed. What's the chip used in the Pi 4?
Comment 22 Fabian Vogt 2024-04-16 12:05:41 UTC
On SP4 at least, the test passes without kernel-firmware-bluetooth installed: https://openqa.suse.de/tests/10599372#step/rpi_bluetooth/13

So I suspect it's an issue with bluez or the kernel.
Comment 23 Zaoliang Luo 2024-04-16 13:18:34 UTC
(In reply to Fabian Vogt from comment #21)
> (In reply to Fabian Vogt from comment #16)
> > Is kernel-firmware-bluetooth really required? So far the needed firmware
> > files for bluetooth were either in kernel-firmware-brcm or bcm43xx-firmware.
> 
> ^
> 
> If kernel-firmware-bluetooth is really needed, it's possible to check which
> file(s) in particular are needed. What's the chip used in the Pi 4?


localhost:~ # cat /sys/firmware/devicetree/base/model
Raspberry Pi 4 Model B Rev 1.1

localhost:~ # cat /proc/cpuinfo
processor	: 0
BogoMIPS	: 108.00
Features	: fp asimd evtstrm crc32 cpuid
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

processor	: 1
BogoMIPS	: 108.00
Features	: fp asimd evtstrm crc32 cpuid
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

processor	: 2
BogoMIPS	: 108.00
Features	: fp asimd evtstrm crc32 cpuid
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

processor	: 3
BogoMIPS	: 108.00
Features	: fp asimd evtstrm crc32 cpuid
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3
Comment 24 Andreas Färber 2024-04-16 13:27:50 UTC
(In reply to Zaoliang Luo from comment #23)
> (In reply to Fabian Vogt from comment #21)
> > If kernel-firmware-bluetooth is really needed, it's possible to check which
> > file(s) in particular are needed. What's the chip used in the Pi 4?
> 
> localhost:~ # cat /sys/firmware/devicetree/base/model
> Raspberry Pi 4 Model B Rev 1.1
> 
> localhost:~ # cat /proc/cpuinfo
[snip]

That's the SoC chipset and its CPUs, not the Bluetooth chipset.
Wifi firmware says it's BCM43455 and it used to be Wifi+BT combo.

https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/brcm
Comment 25 Andreas Färber 2024-04-16 13:32:34 UTC
Lacking other input here, I'd say let's add that firmware package please.

We used to document how to use Bluetooth (thinking keyboard/mouse, for example), so let's keep it usable out-of-the-box for SLES on RPi.

Reassigning to adjust the Kiwi template - note this package is only needed for RPi, not for the aarch64 KVM image.
Comment 27 Zaoliang Luo 2024-04-16 14:07:44 UTC
(In reply to Andreas Färber from comment #24)
> (In reply to Zaoliang Luo from comment #23)
> > (In reply to Fabian Vogt from comment #21)
> > > If kernel-firmware-bluetooth is really needed, it's possible to check which
> > > file(s) in particular are needed. What's the chip used in the Pi 4?
> > 
> > localhost:~ # cat /sys/firmware/devicetree/base/model
> > Raspberry Pi 4 Model B Rev 1.1
> > 
> > localhost:~ # cat /proc/cpuinfo
> [snip]
> 
> That's the SoC chipset and its CPUs, not the Bluetooth chipset.
> Wifi firmware says it's BCM43455 and it used to be Wifi+BT combo.
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/
> tree/brcm

well, what I found in hwinfo:


  P: /devices/platform/soc/fe201000.serial/serial0/serial0-0
  M: serial0-0
  R: 0
  U: serial
  V: hci_uart_bcm
  E: DEVPATH=/devices/platform/soc/fe201000.serial/serial0/serial0-0
  E: SUBSYSTEM=serial
  E: DRIVER=hci_uart_bcm
  E: OF_NAME=bluetooth
  E: OF_FULLNAME=/soc/serial@7e201000/bluetooth
  E: OF_COMPATIBLE_0=brcm,bcm43438-bt
  E: OF_COMPATIBLE_N=1 
  E: MODALIAS=of:NbluetoothT(null)Cbrcm,bcm43438-bt
  
  P: /devices/platform/soc/fe201000.serial/serial0/serial0-0/bluetooth/hci0
  M: hci0
  R: 0
  U: bluetooth
  T: host
  E: DEVPATH=/devices/platform/soc/fe201000.serial/serial0/serial0-0/bluetooth/hci0
  E: SUBSYSTEM=bluetooth
  E: DEVTYPE=host 
  E: USEC_INITIALIZED=27228430
  E: SYSTEMD_ALIAS=/sys/subsystem/bluetooth/devices/hci0
  E: SYSTEMD_WANTS=bluetooth.target
  E: SYSTEMD_USER_WANTS=bluetooth.target
  E: TAGS=:systemd: 
  E: CURRENT_TAGS=:systemd:
  
  P: /devices/platform/soc/fe201000.serial/serial0/serial0-0/bluetooth/hci0/rfkill0
  M: rfkill0
  R: 0
  U: rfkill
  E: DEVPATH=/devices/platform/soc/fe201000.serial/serial0/serial0-0/bluetooth/hci0/rfkill0
  E: SUBSYSTEM=rfkill 
  E: RFKILL_NAME=hci0
  E: RFKILL_TYPE=bluetooth
  E: RFKILL_STATE=1
  E: RFKILL_HW_BLOCK_REASON=0x0
  E: USEC_INITIALIZED=27228588
  E: SYSTEMD_RFKILL=1
  E: ID_PATH=platform-soc-amba-fe201000.serial
  E: ID_PATH_TAG=platform-soc-amba-fe201000_serial
Comment 28 Zaoliang Luo 2024-04-16 14:19:12 UTC
(In reply to Zaoliang Luo from comment #27)
> (In reply to Andreas Färber from comment #24)
> > (In reply to Zaoliang Luo from comment #23)
> > > (In reply to Fabian Vogt from comment #21)
> > > > If kernel-firmware-bluetooth is really needed, it's possible to check which
> > > > file(s) in particular are needed. What's the chip used in the Pi 4?
> > > 
> > > localhost:~ # cat /sys/firmware/devicetree/base/model
> > > Raspberry Pi 4 Model B Rev 1.1
> > > 
> > > localhost:~ # cat /proc/cpuinfo
> > [snip]
> > 
> > That's the SoC chipset and its CPUs, not the Bluetooth chipset.
> > Wifi firmware says it's BCM43455 and it used to be Wifi+BT combo.
> > 
> > https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/
> > tree/brcm
> 
> well, what I found in hwinfo:
> 
> 
>   P: /devices/platform/soc/fe201000.serial/serial0/serial0-0
>   M: serial0-0
>   R: 0
>   U: serial
>   V: hci_uart_bcm
>   E: DEVPATH=/devices/platform/soc/fe201000.serial/serial0/serial0-0
>   E: SUBSYSTEM=serial
>   E: DRIVER=hci_uart_bcm
>   E: OF_NAME=bluetooth
>   E: OF_FULLNAME=/soc/serial@7e201000/bluetooth
>   E: OF_COMPATIBLE_0=brcm,bcm43438-bt
>   E: OF_COMPATIBLE_N=1 
>   E: MODALIAS=of:NbluetoothT(null)Cbrcm,bcm43438-bt
>   
>   P: /devices/platform/soc/fe201000.serial/serial0/serial0-0/bluetooth/hci0
>   M: hci0
>   R: 0
>   U: bluetooth
>   T: host
>   E:
> DEVPATH=/devices/platform/soc/fe201000.serial/serial0/serial0-0/bluetooth/
> hci0
>   E: SUBSYSTEM=bluetooth
>   E: DEVTYPE=host 
>   E: USEC_INITIALIZED=27228430
>   E: SYSTEMD_ALIAS=/sys/subsystem/bluetooth/devices/hci0
>   E: SYSTEMD_WANTS=bluetooth.target
>   E: SYSTEMD_USER_WANTS=bluetooth.target
>   E: TAGS=:systemd: 
>   E: CURRENT_TAGS=:systemd:
>   
>   P:
> /devices/platform/soc/fe201000.serial/serial0/serial0-0/bluetooth/hci0/
> rfkill0
>   M: rfkill0
>   R: 0
>   U: rfkill
>   E:
> DEVPATH=/devices/platform/soc/fe201000.serial/serial0/serial0-0/bluetooth/
> hci0/rfkill0
>   E: SUBSYSTEM=rfkill 
>   E: RFKILL_NAME=hci0
>   E: RFKILL_TYPE=bluetooth
>   E: RFKILL_STATE=1
>   E: RFKILL_HW_BLOCK_REASON=0x0
>   E: USEC_INITIALIZED=27228588
>   E: SYSTEMD_RFKILL=1
>   E: ID_PATH=platform-soc-amba-fe201000.serial
>   E: ID_PATH_TAG=platform-soc-amba-fe201000_serial

should be:

localhost:/sys/firmware/devicetree/base/soc/serial@7e201000 # cat bluetooth/compatible 


brcm,bcm43438-bt
Comment 32 Fabian Vogt 2024-04-17 07:48:14 UTC
If https://github.com/RPi-Distro/bluez-firmware/commit/3410dbbdc124927874e67ea2592a8bc63660a7d7 is correct, the right firmware file is BCM4345C0.hcd, part of the bcm43xx-firmware package.
Comment 33 Joey Lee 2024-04-17 08:42:54 UTC
After d883a4669a be introduced to v6.4 kernel, the hci_cmd_sync_queue always checks HCI_RUNNING tag:

commit d883a4669a1def6d121ccf5e64ad28260d1c9531                 [v6.4-rc1]
Author: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
Date:   Fri Apr 21 11:37:55 2023 -0700

    Bluetooth: hci_sync: Only allow hci_cmd_sync_queue if running

    This makes sure hci_cmd_sync_queue only queue new work if HCI_RUNNING
    has been set otherwise there is a risk of commands being sent while
    turning off.

    Because hci_cmd_sync_queue can no longer queue work while HCI_RUNNING is
    not set it cannot be used to power on adapters so instead
    hci_cmd_sync_submit is introduced which bypass the HCI_RUNNING check, so
    it behaves like the old implementation.

When booting is finished, the bluetooth controller be set to powered-off state. So HCI_RUNNING be cleared. It's OK because user can use bluetoothctl to power-on controller when he wants to use bluetooth functions.

The point is that user runs stop/start bluetooth deamon after booting. The reset_adv_monitors_complete() Failed shows up because it tried to reset Adv Monitors when the bluetooth controller be set power-off. In default behavior, bluetooth daemon does not auto power-on controller. But d883a4669a1 kernel patch checked HCI_RUNNING=0 in hci_cmd_sync_queue(). So the reset-adv-monitors action is blocked.

Reassigned this issue to me because this bug doesn't relates to kernel-firmware-bluetooth. It's just a coincidence because we need reboot after kernel-firmware-bluetooth is installed. The bluetooth daemon doesn't have this problem when booting.

On the other hand, the remove_adv_monitor() also be run when booting. So the reset-adv-monitors action failed when restart bluetooth daemon is redundant action. I don't think it's a serious problem.
Comment 34 Joey Lee 2024-04-17 16:20:53 UTC
This issue can be reproduced on x86_64 machines with:

- kernel 6.6.6-1-default
- bluez-5.71-3.3.x86_64           

The behavior is the same. If bluetooth daemon be enabled and start when booting, no problem. If bluetooth daemon be stop/start after booting, then reset_adv_monitors_complete() message shows up.
Comment 35 Joey Lee 2024-04-18 09:59:27 UTC
(In reply to Joey Lee from comment #34)
> This issue can be reproduced on x86_64 machines with:
> 
> - kernel 6.6.6-1-default
> - bluez-5.71-3.3.x86_64           
> 
> The behavior is the same. If bluetooth daemon be enabled and start when
> booting, no problem. If bluetooth daemon be stop/start after booting, then
> reset_adv_monitors_complete() message shows up.

The issue can be produced with kernel v6.9.0 + bluez 5.75. It's a race condition issue between MGMT_OP_REMOVE_ADV_MONITOR command with power of controller.

Base on doc/mgmt-api.txt document, the MGMT_OP_REMOVE_ADV_MONITOR command can be used when controller is power-off:

Remove Advertisement Monitor Command                    
====================================

        Command Code:           0x0053
        Controller Index:       <controller id>
        Command Parameters:     Monitor_Handle (2 Octets)
        Return Parameters:      Monitor_Handle (2 Octets)

        This command is used to remove advertisement monitor(s). The kernel     
        would remove the monitor(s) with Monitor_Handle and update the LE       
        scanning.

        When the Monitor_Handle is set to zero, then all previously added       
        handles will be removed.

        Removing a monitor while it is being added will be ignored.             

        This command can be used when the controller is not powered and         
        all settings will be programmed once powered.

        Possible errors:        Failed
                                Busy

Looks that kernel should queues MGMT_OP_REMOVE_ADV_MONITOR and running it when power-on. But after d883a4669a1 be introduced in v6.4 kernel, MGMT_OP_REMOVE_ADV_MONITOR command got failed from kernel.
Comment 36 Jan Stehlik 2024-04-18 14:30:05 UTC
Blocker flag removed since the workaround allows for bluetooth testing.
Comment 39 Radoslav Tzvetkov 2024-04-24 12:56:17 UTC
Hello Andreas, please provide your decision about the further actions on this bug regarding the upcoming SP6 Gold Master.

Our submission deadline for the code is 15.05, so we need your decision ASAP.
Comment 40 Joey Lee 2024-04-25 09:38:12 UTC
I have sent patch to kernel mainline for reviewing:

https://lore.kernel.org/lkml/20240424152319.GD26307@linux-l9pv.suse/T/

I will backport the patch to 15-SP6 first, before grabbing any concern from upstream.
Comment 42 Joey Lee 2024-04-25 10:04:49 UTC
(In reply to Joey Lee from comment #40)
> I have sent patch to kernel mainline for reviewing:
> 
> https://lore.kernel.org/lkml/20240424152319.GD26307@linux-l9pv.suse/T/
> 
> I will backport the patch to 15-SP6 first, before grabbing any concern from
> upstream.

Backported patch to sent to 15-SP6-GA/for-next and wait to be merged.
Comment 43 Joey Lee 2024-04-25 19:52:54 UTC
(In reply to Joey Lee from comment #42)
> (In reply to Joey Lee from comment #40)
> > I have sent patch to kernel mainline for reviewing:
> > 
> > https://lore.kernel.org/lkml/20240424152319.GD26307@linux-l9pv.suse/T/
> > 
> > I will backport the patch to 15-SP6 first, before grabbing any concern from
> > upstream.
> 
> Backported patch to sent to 15-SP6-GA/for-next and wait to be merged.

Backported patch be merged to 15-SP6-GA kernel.
Comment 46 Joey Lee 2024-05-03 04:39:33 UTC
(In reply to Joey Lee from comment #40)
> I have sent patch to kernel mainline for reviewing:
> 
> https://lore.kernel.org/lkml/20240424152319.GD26307@linux-l9pv.suse/T/
> 
> I will backport the patch to 15-SP6 first, before grabbing any concern from
> upstream.

The patch be merged to mainline:

commit 88cd6e6b2d327faa13e4505b07f1e380e51b21ff
Author: Chun-Yi Lee <jlee@suse.com>
Date:   Wed Apr 24 21:59:03 2024 +0800

    Bluetooth: hci_sync: Using hci_cmd_sync_submit when removing Adv Monitor
Comment 48 Radoslav Tzvetkov 2024-05-03 12:53:17 UTC
Resolved Fixed in Build 83.1