Bug 1226544 - Snapshot 20240618, kernel 6.9.5, breaks Intel AX210 WiFi connection / kernel-firmware-iwlwifi issue
Summary: Snapshot 20240618, kernel 6.9.5, breaks Intel AX210 WiFi connection / kernel-...
Status: RESOLVED FIXED
: 1226667 1226689 1226717 1226779 1226978 (view as bug list)
Alias: None
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Kernel:Networking (show other bugs)
Version: Current
Hardware: 64bit openSUSE Tumbleweed
: P5 - None : Major (vote)
Target Milestone: ---
Assignee: Kernel Bugs
QA Contact: E-mail List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-06-20 02:32 UTC by Archer Allstars
Modified: 2024-06-29 16:17 UTC (History)
27 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments
dmesg log with kernel 6.9.5 (693.52 KB, text/plain)
2024-06-20 06:53 UTC, Steven Braun
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Archer Allstars 2024-06-20 02:32:49 UTC
After updating to Tumbleweed snapshot 20240818 with kernel 6.9.5, I can't connect with my WiFi. 

- The card is Intel AX210.
- The driver of the card is iwlwifi.

If further log is needed, please tell me which log, and how to get it. Thanks.
Comment 1 Archer Allstars 2024-06-20 02:52:36 UTC
EDIT: It's snapshot 20240618.
Comment 2 Steven Braun 2024-06-20 06:53:09 UTC
Created attachment 875587 [details]
dmesg log with kernel 6.9.5

I can confirm this issue with kernel 6.9.5. Attached is the dmesg log after booting into 6.9.5.
Comment 3 Philippe Schottey 2024-06-20 07:02:29 UTC
I have the same issue with:

Network controller: Intel Corporation Alder Lake-P PCH CNVi WiFi (rev 01)
Subsystem: Intel Corporation Device 0090
Kernel driver in use: iwlwifi
Kernel modules: iwlwifi
Comment 4 Philippe Schottey 2024-06-20 07:06:24 UTC
EDIT: device seems to be Intel(R) Wi-Fi 6E AX211
Comment 5 Petr Valenta 2024-06-20 08:04:33 UTC
There is a problem with kernel-firmware-iwlwifi. 

Workaround:
1) reboot to previous working snapshot
2) snapper rollback
3) zypper al kernel-firmware-iwlwifi
4) zypper dup .... and select option to keep obsolete firmware packages

Now wifi works in latest (6.9.5-1-default) kernel.
Comment 6 Takashi Iwai 2024-06-20 10:19:43 UTC
Is it the update of kernel-default 6.9.x or the kernel-firmware-* package?

You can find the older TW kernels in history repo
  http://download.opensuse.org/history/

and try to downgrade only kernel-default or kernel-firmware-*, and identify which one is broken.
Comment 7 Tobias Burnus 2024-06-20 10:43:03 UTC
If I got to:
http://download.opensuse.org/history/20240618/tumbleweed/repo/oss/noarch/

and download
kernel-firmware-iwlwifi-20240519-1.1.noarch.rpm (22.5.2024, 21:54, 58.1 MB)
and then do:

rpm -e --nodeps kernel-firmware-iwlwifi-20240618-1.1.noarch
rpm -Uvvh --nodeps kernel-firmware-iwlwifi-20240519-1.1.noarch.rpm

and reboot, wifi seems to work fine with 6.9.5-1-default.
Comment 8 Takashi Iwai 2024-06-20 11:32:56 UTC
Thanks!  Then it's a firmware regression.  I'll report to Intel.
Comment 9 Mikhail Paulyshka 2024-06-20 15:18:22 UTC
(In reply to Takashi Iwai from comment #8)
> Thanks!  Then it's a firmware regression.  I'll report to Intel.

Maybe we can roll back the broken firmware while Intel prepares a fix?

https://build.opensuse.org/request/show/1181983
Comment 10 Takashi Iwai 2024-06-20 15:21:00 UTC
Judging from the upstream response, it sounds like that 6.9.6 kernel will work with the new firmware.  But he didn't reply to my confirmation, so I'm not 100% sure, though.
Comment 11 Mikhail Paulyshka 2024-06-20 15:29:16 UTC
(In reply to Takashi Iwai from comment #10)
> Judging from the upstream response, it sounds like that 6.9.6 kernel will
> work with the new firmware.  But he didn't reply to my confirmation, so I'm
> not 100% sure, though.

Looking at the dmesg log 

```
[  317.092491] [ T2126] iwlwifi 0000:03:00.0: FW error in SYNC CMD REDUCE_TX_POWER_CMD
```

and commit description
```
the firmware hit a BAD_COMMAND immediately after handling the REDUCE_TX_POWER_CMD command.
```

it could be
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=788e4c75f831d06fcfbbec1d455fac429521e607
Comment 12 Takashi Iwai 2024-06-20 15:36:24 UTC
That's my doubt, too.  But interestingly, this commit has
  Fixes: 8f892e225f41 ("wifi: iwlwifi: mvm: support iwl_dev_tx_power_cmd_v8")
but this commit hasn't been backported in 6.9.5.  This commit is backported to 6.9.6 together with the fix above.

In anyway, those commits should have been already included in 6.10-rc3, so the kernel in OBS Kernel:HEAD kernel should work.  Can anyone check it?
Comment 13 Mikhail Paulyshka 2024-06-20 15:56:09 UTC
(In reply to Takashi Iwai from comment #12)
> That's my doubt, too.  But interestingly, this commit has
>   Fixes: 8f892e225f41 ("wifi: iwlwifi: mvm: support iwl_dev_tx_power_cmd_v8")
> but this commit hasn't been backported in 6.9.5.  This commit is backported
> to 6.9.6 together with the fix above.
> 
> In anyway, those commits should have been already included in 6.10-rc3, so
> the kernel in OBS Kernel:HEAD kernel should work.  Can anyone check it?

6.10.0-rc4-1.g3306b36-default works just fine!

---

Trying to check 6.9.5 + 788e4c75f831d06fcfbbec1d455fac429521e607
Comment 14 Mikhail Paulyshka 2024-06-20 18:08:02 UTC
(In reply to Mikhail Paulyshka from comment #13)
> (In reply to Takashi Iwai from comment #12)
> > That's my doubt, too.  But interestingly, this commit has
> >   Fixes: 8f892e225f41 ("wifi: iwlwifi: mvm: support iwl_dev_tx_power_cmd_v8")
> > but this commit hasn't been backported in 6.9.5.  This commit is backported
> > to 6.9.6 together with the fix above.
> > 
> > In anyway, those commits should have been already included in 6.10-rc3, so
> > the kernel in OBS Kernel:HEAD kernel should work.  Can anyone check it?
> 
> 6.10.0-rc4-1.g3306b36-default works just fine!
> 
> ---
> 
> Trying to check 6.9.5  + 788e4c75f831d06fcfbbec1d455fac429521e607

It also works fine with
 - 6.9.5 
 + 8f892e225f416fcf2b55a0f9161162e08e2b0cc7 (pre-requirement)
 + 788e4c75f831d06fcfbbec1d455fac429521e607 (actual fix)


Request: https://build.opensuse.org/request/show/1182002
Comment 15 Mikhail Paulyshka 2024-06-20 18:12:09 UTC
Upstream bugreport https://bugzilla.kernel.org/show_bug.cgi?id=218963

Looks like Intel AX211 is also affected.
Comment 16 Arjen de Korte 2024-06-20 20:28:25 UTC
> Looks like Intel AX211 is also affected.

I can confirm that AX211 is indeed affected.
Comment 17 Philippe Schottey 2024-06-21 06:02:31 UTC
So what is the advice here? Just wait on the next opensuse release. Or must I absolutely install this release and fix it with the suggested workaround? 

Failrly new on opensuse tumebleweed, hence the question.
Comment 18 Tobias Burnus 2024-06-21 06:45:01 UTC
(In reply to Mikhail Paulyshka from comment #14)
> Request: https://build.opensuse.org/request/show/1182002

I have no deeper insight into how the (open)SUSE's kernels are developed, but shouldn't that be a rather a pull request against
https://github.com/openSUSE/kernel-source for branch 'stable'?
Comment 19 Takashi Iwai 2024-06-21 07:44:24 UTC
I sent a PR to stable git branch to include those two commit backports.
It'll be included in the next update, or possibly together with 6.9.6 update.

Until the kernel fix appears, you can download to the old kernel-firmware-iwlwifi package from TW history repo:
  http://download.opensuse.org/history/20240618/tumbleweed/repo/oss/
Comment 20 Takashi Iwai 2024-06-21 08:33:15 UTC
*** Bug 1226667 has been marked as a duplicate of this bug. ***
Comment 21 hui 2024-06-21 08:43:49 UTC
*** Bug 1226689 has been marked as a duplicate of this bug. ***
Comment 22 Oleg Antonyan 2024-06-21 12:40:13 UTC
*** Bug 1226717 has been marked as a duplicate of this bug. ***
Comment 23 Tobias Burnus 2024-06-21 13:42:51 UTC
> I sent a PR to stable git branch to include those two commit backports.

Which has been merged: https://github.com/openSUSE/kernel-source/commits/stable

It seems as if https://build.opensuse.org/package/show/Kernel:stable/kernel-source automatically gets updated daily at 6am UTC, but can also be triggered manually.
@tiwai - Can this be done + then a request for Tumbleweed be opened?


BTW: 6.9.6 has been now released:
https://lore.kernel.org/lkml/2024062120-quilt-qualified-d0dd@gregkh/
Comment 24 Mikhail Paulyshka 2024-06-21 13:50:05 UTC
*** Bug 1226779 has been marked as a duplicate of this bug. ***
Comment 25 Takashi Iwai 2024-06-21 14:30:40 UTC
The OBS kernel build will happen automatically in tomorrow, yes.

For the TW submision: the submission is done usually by Jiri.

Jiri, as this seems hitting many users, could you submit the current stable git repo to TW?  TIA!
Comment 26 Philippe Schottey 2024-06-21 14:39:03 UTC
Can someone explain to me how the tumbleweed update will happen? Will the next update automatically contain the fix? Or do we need to install the 20240618 first to get the next one?

I currently reverted to 20240617.
Comment 27 Muzhi Tian 2024-06-22 01:46:32 UTC
(In reply to Philippe Schottey from comment #26)
> Will the next update automatically contain the fix?
> Or do we need to install the  20240618 first to get the next one?

zypper dup, without any changes to the default repo configuration, will pull from the latest snapshot, no need to download anything in between. I sometimes go weeks between updates, the usual dist-upgrade will get me on the latest everything every time (my excuse this time is that I'm migrating to a new system but actually I'm just lazy).
Comment 28 Alexander van Kaam 2024-06-22 15:19:14 UTC
Also affecting the Wi-Fi 6E AX1675w (210D2W), rolled back for now
Comment 29 Frank Krüger 2024-06-22 15:59:43 UTC
(In reply to Alexander van Kaam from comment #28)
> Also affecting the Wi-Fi 6E AX1675w (210D2W), rolled back for now

Maybe it's worth a try: The above-mentioned issue seems to be fixed in kernel 6.9.5-6.1 from Kernel:stable repo and upstream in version 6.9.6 (https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.9.66.1.gb32a843.x86_64).
Comment 30 Frank Krüger 2024-06-22 16:09:34 UTC
(In reply to Frank Krüger from comment #29)
> (In reply to Alexander van Kaam from comment #28)
> > Also affecting the Wi-Fi 6E AX1675w (210D2W), rolled back for now
> 
> Maybe it's worth a try: The above-mentioned issue seems to be fixed in
> kernel 6.9.5-6.1 from Kernel:stable repo and upstream in version 6.9.6
> (https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.9.66.1.gb32a843.
> x86_64).

Sorry, the correct upstream link should read: https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.9.6
Comment 31 Jiri Slaby 2024-06-24 05:40:10 UTC
(In reply to Takashi Iwai from comment #25) 
> Jiri, as this seems hitting many users, could you submit the current stable
> git repo to TW?  TIA!

Done as:
https://build.opensuse.org/request/show/1182944
Comment 32 hui 2024-06-25 20:35:13 UTC
*** Bug 1226978 has been marked as a duplicate of this bug. ***
Comment 33 Archer Allstars 2024-06-27 06:50:56 UTC
This issue is fixed for me (AX210) on snapshot 20240625, kernel 6.9.6. As there are many cards affected, feel free to close this issue as needed.

Thanks!
Comment 34 Takashi Iwai 2024-06-27 08:25:12 UTC
As 6.9.6 seems working, let's close.
Comment 35 Felix Niederwanger 2024-06-27 09:43:05 UTC
Confirmed fixed also here, AX210 Kernel 6.9.6-1, TW snapshot 20240625. Thank you eveyone!
Comment 36 Zala 2024-06-27 12:57:07 UTC
Fixed AX211 with Kernel 6.9.6-1, TW snapshot 20240625. Thank you everyone!
Comment 37 Alexander van Kaam 2024-06-27 14:44:57 UTC
And AX1675w also fixed, thank you
Comment 38 Dan Yeaw 2024-06-29 16:01:13 UTC
I have an Intel AX210/AX1675* 2x2 [Typhoon Peak] (rev 1a) adapter, and I am still seeing this issue even after snapshot 20240627.
Comment 39 Dan Yeaw 2024-06-29 16:17:37 UTC
Sorry for the noise, I rolled back and updated the snapshot again and things are now working.