Bug 1219542

Summary: kernel boot problem after 6.7.* on SAS1068E
Product: [openSUSE] openSUSE Tumbleweed Reporter: Dmitry Markov <werwolf131313>
Component: Kernel:StorageAssignee: Kernel Bugs <kernel-bugs>
Status: NEW --- QA Contact: E-mail List <qa-bugs>
Severity: Major    
Priority: P5 - None CC: tiwai, werwolf131313
Version: Current   
Target Milestone: ---   
Hardware: x86-64   
OS: openSUSE Tumbleweed   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: kernel panic

Description Dmitry Markov 2024-02-04 15:34:12 UTC 
Created attachment 872437 [details]
kernel panic

after upgrade kernel from 6.6.11 to 6.7.2 system can't boot
i add kernel panic photo as attachment

> [werwolf@power] ~  
> ❯ cat /etc/fstab
> # device                                   path                            fs     options                   dump pass
> 
> # block device
> LABEL=ost_root                             /                               btrfs  compress=zstd,discard     0  0
> LABEL=ost_root                             /.snapshots                     btrfs  subvol=/@/.snapshots      0  0
> LABEL=ost_home                             /home                           btrfs  compress=zstd,autodefrag  0  0
> LABEL=ost_home                             /var/lib/plexmediaserver        btrfs  subvol=/var/lib/plexmediaserver,compress=zstd,autodefrag  0  0
> UUID=6b2e5908-4406-4cd5-8da1-bf937ab002c0  swap                            swap   defaults                  0  0

> [werwolf@power] ~  
> ❯ sudo lspci              
> 00:00.0 Host bridge: Intel Corporation Core Processor DMI (rev 11)
> 00:03.0 PCI bridge: Intel Corporation Core Processor PCI Express Root Port 1 (rev 11)
> 00:05.0 PCI bridge: Intel Corporation Core Processor PCI Express Root Port 3 (rev 11)
> 00:08.0 System peripheral: Intel Corporation Core Processor System Management Registers (rev 11)
> 00:08.1 System peripheral: Intel Corporation Core Processor Semaphore and Scratchpad Registers (rev 11)
> 00:08.2 System peripheral: Intel Corporation Core Processor System Control and Status Registers (rev 11)
> 00:08.3 System peripheral: Intel Corporation Core Processor Miscellaneous Registers (rev 11)
> 00:10.0 System peripheral: Intel Corporation Core Processor QPI Link (rev 11)
> 00:10.1 System peripheral: Intel Corporation Core Processor QPI Routing and Protocol Registers (rev 11)
> 00:1a.0 USB controller: Intel Corporation 5 Series/3400 Series Chipset USB2 Enhanced Host Controller (rev 05)
> 00:1c.0 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 1 (rev 05)
> 00:1c.5 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 6 (rev 05)
> 00:1c.6 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 7 (rev 05)
> 00:1c.7 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 8 (rev 05)
> 00:1d.0 USB controller: Intel Corporation 5 Series/3400 Series Chipset USB2 Enhanced Host Controller (rev 05)
> 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a5)
> 00:1f.0 ISA bridge: Intel Corporation 3420 Chipset LPC Interface Controller (rev 05)
> 00:1f.2 SATA controller: Intel Corporation 5 Series/3400 Series Chipset 6 port SATA AHCI Controller (rev 05)
> 00:1f.3 SMBus: Intel Corporation 5 Series/3400 Series Chipset SMBus Controller (rev 05)
> 01:00.0 SCSI storage controller: Broadcom / LSI SAS1068E PCI-Express Fusion-MPT SAS (rev 08)
> 0f:00.0 PCI bridge: Microsemi / PMC / IDT PES12N3A 12-lane 3-Port PCI Express Switch (rev 0e)
> 10:02.0 PCI bridge: Microsemi / PMC / IDT PES12N3A 12-lane 3-Port PCI Express Switch (rev 0e)
> 10:04.0 PCI bridge: Microsemi / PMC / IDT PES12N3A 12-lane 3-Port PCI Express Switch (rev 0e)
> 11:00.0 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06)
> 11:00.1 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06)
> 12:00.0 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06)
> 12:00.1 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06)
> 13:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
> 14:00.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200e [Pilot] ServerEngines (SEP1) (rev 02)
> ff:00.0 Host bridge: Intel Corporation Core Processor QuickPath Architecture Generic Non-Core Registers (rev 04)
> ff:00.1 Host bridge: Intel Corporation Core Processor QuickPath Architecture System Address Decoder (rev 04)
> ff:02.0 Host bridge: Intel Corporation Core Processor QPI Link 0 (rev 04)
> ff:02.1 Host bridge: Intel Corporation Core Processor QPI Physical 0 (rev 04)
> ff:03.0 Host bridge: Intel Corporation Core Processor Integrated Memory Controller (rev 04)
> ff:03.1 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Target Address Decoder (rev 04)
> ff:03.2 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Test Registers (rev 04)
> ff:03.4 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Test Registers (rev 04)
> ff:04.0 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 0 Control Registers (rev 04)
> ff:04.1 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 0 Address Registers (rev 04)
> ff:04.2 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 0 Rank Registers (rev 04)
> ff:04.3 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 0 Thermal Control Registers (rev 04)
> ff:05.0 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 1 Control Registers (rev 04)
> ff:05.1 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 1 Address Registers (rev 04)
> ff:05.2 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 1 Rank Registers (rev 04)
> ff:05.3 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 1 Thermal Control Registers (rev 04)

> System:
>   Host: power.itrus.su Kernel: 6.6.11-1-default arch: x86_64 bits: 64
>     compiler: gcc v: 13.2.1 clocksource: tsc
>   Desktop: N/A wm: KWin dm: SDDM Distro: openSUSE Tumbleweed 20240201
> Machine:
>   Type: Server System: FUJITSU product: PRIMERGY TX150 S7 v: GS01
>     serial: <superuser required> Chassis: type: 17 v: TX150S7FS
>     serial: <superuser required>
>   Mobo: FUJITSU model: D2759 v: S26361-D2759-A13 WGS04 GS02
>     serial: <superuser required> part-nu: ABN:K1265-V101-88
>     uuid: <superuser required> BIOS: FUJITSU // Phoenix
>     v: 6.00 Rev. 1.21.2759.A1 date: 07/11/2018

> [werwolf@power] ~  
> ❯ LANG=c zypper info kernel-default
> 
> Information for package kernel-default:
> ---------------------------------------
> Repository     : repo-oss
> Name           : kernel-default
> Version        : 6.7.2-1.2
> Arch           : x86_64
> Vendor         : openSUSE
> Installed Size : 240.4 MiB
> Installed      : Yes
> Status         : out-of-date (version 6.7.2-1.1 installed)
> Source package : kernel-default-6.7.2-1.2.nosrc
> Upstream URL   : https://www.kernel.org/
> Summary        : The Standard Kernel
> Description    : 
>     The standard kernel for both uniprocessor and multiprocessor systems.
> 
> 
>     Source Timestamp: 2024-01-26 11:01:28 +0000
>     GIT Revision: a52bf765724a419339d0b0ac309ebfbac92845a8
>     GIT Branch: stable

Previously (probably about a year ago) the system already refused to boot when using the UUID in fstab, I changed it to LABEL, but after a while the problem returned
Comment 1 Takashi Iwai 2024-03-11 16:03:57 UTC 
Do you still see the similar crashes with the latest 6.7.x kernel?

TW is already moving from 6.7.x to 6.8, it'd be worth to check the 6.8.0 kernel in OBS Kernel:stable repo, too.
Comment 2 Dmitry Markov 2024-03-11 16:51:20 UTC 
(In reply to Takashi Iwai from comment #1)
> Do you still see the similar crashes with the latest 6.7.x kernel?
> 
> TW is already moving from 6.7.x to 6.8, it'd be worth to check the 6.8.0
> kernel in OBS Kernel:stable repo, too.

It took me some time to check this, despite the fact that this is a home server and it is used by a considerable number of people using my services.

> [werwolf@power] ~  
> ❯ uname -a
> Linux power.itrus.su 6.7.7-1-default #1 SMP PREEMPT_DYNAMIC Fri Mar  1 13:51:21 UTC 2024 (1ff84c5) x86_64 x86_64 x86_64 GNU/Linux

future version works too

> [werwolf@power] ~  
> ❯ uname -a
> Linux power.itrus.su 6.8.0-2.ga00a1ae-default #1 SMP PREEMPT_DYNAMIC Mon Mar 11 09:45:07 UTC 2024 (a00a1ae) x86_64 x86_64 x86_64 GNU/Linux

It’s hard to describe in words how glad I am that I can remove the lock from the kernel-default package. although I still didn’t understand what the problem was.