Bug 1219973

Summary:	15 SP6 baremetal installation fail and system will not reboot
Product:	[openSUSE] PUBLIC SUSE Linux Enterprise Server 15 SP6	Reporter:	Petr Cervinka <pcervinka>
Component:	X.Org	Assignee:	Gfx Enterprise Bugs <gfx-enterprise-bugs>
Status:	RESOLVED DUPLICATE	QA Contact:	Gfx Bugs <gfx-bugs>
Severity:	Normal
Priority:	P5 - None	CC:	jalausuch, mmoese, okurz
Version:	unspecified
Target Milestone:	---
Hardware:	Other
OS:	Other
URL:	https://openqa.suse.de/tests/13493639/modules/handle_reboot/steps/2
Whiteboard:
Found By:	openQA	Services Priority:
Business Priority:		Blocker:	Yes
Marketing QA Status:	---	IT Deployment:	---
Attachments:	yast logs from isntallation ipmi console log VLC screenshot from just before the hexdump started

Description Petr Cervinka 2024-02-15 13:53:57 UTC

Created attachment 872766 [details]
yast logs from isntallation

## Observation

openQA test in scenario sle-15-SP6-Online-x86_64-prepare_baremetal@ipmi-coppi-xen fails in
[handle_reboot](https://openqa.suse.de/tests/13493639/modules/handle_reboot/steps/2)

## Test suite description
Maintainer: pcervinka. Proceed with installation on bare metal machines.



## Reproducible

Fails since (at least) Build [46.40](https://openqa.suse.de/tests/13242238)


## Expected result

Last good: (unknown) (or more recent)


## Further details

Always latest result in this scenario: [latest](https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Online&machine=ipmi-coppi-xen&test=prepare_baremetal&version=15-SP6)


Issue is reproducible on multiple x86_64 baremetal machines and even on aarch64 machine.

Problem starts when installation is finished and we are supposed to click ok on OK to reboot system.  System will not be restarted and console (in this case raw  ipmi console) is flooded with trash characters.

It is still possible to connect to "installer" system and collect logs. There is no obvious error.

I'm attaching yast logs + console logs collected directly from ipmi console.

Comment 1 Petr Cervinka 2024-02-15 13:54:22 UTC

Created attachment 872767 [details]
ipmi console log

Comment 2 Stefan Hundhammer 2024-02-15 14:26:33 UTC

From that console log:

> *** Starting YaST ***
> WARNING: Nokogiri was built against LibXML version 2.9.14, but has dynamically loaded 2.10.3
> /mounts/mp_0001/usr/share/YaST2/lib/registration/addon.rb:360: warning: Registration::Addon#eula_url at /mounts/mp_0001/usr/lib64/ruby/2.5.0/forwardable.rb:157 forwarding to private method OpenStruct#eula_url
> /mounts/mp_0001/usr/share/YaST2/lib/registration/addon.rb:360: warning: Registration::Addon#eula_url at /mounts/mp_0001/usr/lib64/ruby/2.5.0/forwardable.rb:157 forwarding to private method OpenStruct#eula_url
> /mounts/mp_0001/usr/share/YaST2/lib/registration/addon.rb:360: warning: Registration::Addon#eula_url at /mounts/mp_0001/usr/lib64/ruby/2.5.0/forwardable.rb:157 forwarding to private method OpenStruct#eula_url
> /mounts/mp_0001/usr/share/YaST2/lib/registration/addon.rb:360: warning: Registration::Addon#eula_url at /mounts/mp_0001/usr/lib64/ruby/2.5.0/forwardable.rb:157 forwarding to private method OpenStruct#eula_url
> /mounts/mp_0001/usr/share/YaST2/lib/registration/addon.rb:360: warning: Registration::Addon#eula_url at /mounts/mp_0001/usr/lib64/ruby/2.5.0/forwardable.rb:157 forwarding to private method OpenStruct#eula_url
> /mounts/mp_0001/usr/share/YaST2/lib/registration/addon.rb:352: warning: Registration::Addon#release_type at /mounts/mp_0001/usr/lib64/ruby/2.5.0/forwardable.rb:157 forwarding to private method OpenStruct#release_type
> /mounts/mp_0001/usr/share/YaST2/lib/registration/addon.rb:352: warning: Registration::Addon#release_type at /mounts/mp_0001/usr/lib64/ruby/2.5.0/forwardable.rb:157 forwarding to private method OpenStruct#release_type
> /mounts/mp_0001/usr/share/YaST2/lib/registration/addon.rb:352: warning: Registration::Addon#release_type at /mounts/mp_0001/usr/lib64/ruby/2.5.0/forwardable.rb:157 forwarding to private method OpenStruct#release_type
> /mounts/mp_0001/usr/share/YaST2/lib/registration/addon.rb:352: warning: Registration::Addon#release_type at /mounts/mp_0001/usr/lib64/ruby/2.5.0/forwardable.rb:157 forwarding to private method OpenStruct#release_type
> /mounts/mp_0001/usr/share/YaST2/lib/registration/addon.rb:352: warning: Registration::Addon#release_type at /mounts/mp_0001/usr/lib64/ruby/2.5.0/forwardable.rb:157 forwarding to private method OpenStruct#release_type
> /mounts/mp_0001/usr/share/YaST2/lib/registration/addon.rb:330: warning: Registration::Addon#release_type at /mounts/mp_0001/usr/lib64/ruby/2.5.0/forwardable.rb:157 forwarding to private method OpenStruct#release_type
> /mounts/mp_0001/usr/share/YaST2/lib/registration/addon.rb:330: warning: Registration::Addon#release_type at /mounts/mp_0001/usr/lib64/ruby/2.5.0/forwardable.rb:157 forwarding to private method OpenStruct#release_type
> /mounts/mp_0001/usr/share/YaST2/lib/registration/addon.rb:330: warning: Registration::Addon#release_type at /mounts/mp_0001/usr/lib64/ruby/2.5.0/forwardable.rb:157 forwarding to private method OpenStruct#release_type
> /mounts/mp_0001/usr/share/YaST2/lib/registration/addon.rb:330: warning: Registration::Addon#release_type at /mounts/mp_0001/usr/lib64/ruby/2.5.0/forwardable.rb:157 forwarding to private method OpenStruct#release_type
> /mounts/mp_0001/usr/share/YaST2/lib/registration/addon.rb:330: warning: Registration::Addon#release_type at /mounts/mp_0001/usr/lib64/ruby/2.5.0/forwardable.rb:157 forwarding to private method OpenStruct#release_type
> 16:07:42 <4> +0x22bb3                    : === extend started ===
> 16:07:43 <4> +0x22bb3                    : === extend started ===
> 16:07:55 <4> +0x22bb3                    : === extend started ===
> 16:07:57 <4> +0x22bb3                    : === extend started ===
> begin 644 Xvnc.core.pid_5414.sig_6.time_1707581851
> M?T5,1@(!`0````````````0`/@`!``````````````!`````````````````
> M`````````$``.`"^``````````0`````````T"D`````````````````````
> M````````\"P``````````````````````````````0````4`````8```````
> M``!`S\Y-5@``````````````$`````````!`)@```````!`````````!````
> M!`````!P`````````(#USDU6``````````````!@`````````&``````````
> M$`````````$````&`````-``````````X/7.358``````````````'``````


So that hexdump appears to be a core dump from Xvnc.

Comment 3 Stefan Hundhammer 2024-02-15 14:33:52 UTC

When I watch the installation video, at 1:25 I see the YaST reboot pop-up, then some messages

Active interfaces:

eth0: ac:1f:6b:65:8d:38
  10.168.192.70/22
  fe80:ae1f:6bff:fe65:8d38/64

and then immediately the hexdump starts scrolling by.

The "Active interfaces" message with the Mac address and the IPv4/Ipv6 addresses are left over on the system console from 0:21 (linuxrc / yast2.start script) just before Xvnc was started at 0:27.

Comment 4 Stefan Hundhammer 2024-02-15 14:36:22 UTC

I am not so sure if this is a YaST problem. More likely, the Xvnc core dump confused the openQA mechanics. Maybe it would still have booted with a little patience; it takes a while until all the Megabytes of the hexdump have scrolled by.

Did you try that; just waiting until maybe it finally rebooted?

Comment 5 Stefan Hundhammer 2024-02-15 14:38:01 UTC

Created attachment 872770 [details]
VLC screenshot from just before the hexdump started

Comment 6 Stefan Hundhammer 2024-02-15 14:41:24 UTC

From vncserver.log of the attached y2logs tarball:

> Thu Feb 15 14:12:48 2024
>  VNCSConnST:  closing 10.149.211.66::45240: Server shutdown
>  EncodeManager: Framebuffer updates: 2679
>  EncodeManager:   Tight:
>  EncodeManager:     Solid: 2.39 krects, 13.0288 Mpixels
>  EncodeManager:            37.3438 KiB (1:1363.59 ratio)
>  EncodeManager:     Bitmap RLE: 446 rects, 124.306 kpixels
>  EncodeManager:                 13.1172 KiB (1:37.4163 ratio)
>  EncodeManager:     Indexed RLE: 4.075 krects, 2.52163 Mpixels
>  EncodeManager:                  592.567 KiB (1:16.7034 ratio)
>  EncodeManager:   Tight (JPEG):
>  EncodeManager:     Full Colour: 2.147 krects, 8.94265 Mpixels
>  EncodeManager:                  9.39714 MiB (1:3.63282 ratio)
>  EncodeManager:   Total: 9.058 krects, 24.6173 Mpixels
>  EncodeManager:          10.0251 MiB (1:9.3776 ratio)
>  ComparingUpdateTracker: 122.173 Mpixels in / 20.2973 Mpixels out
>  ComparingUpdateTracker: (1:6.01917 ratio)
> (EE) 
> (EE) Backtrace:
> (EE) 0: /usr/bin/Xvnc (xorg_backtrace+0x65) [0x55cd52dd9ee5]
> (EE) 1: /usr/bin/Xvnc (0x55cd52c44000+0x199879) [0x55cd52ddd879]
> (EE) 2: /lib64/libc.so.6 (0x7fbdf99a6000+0x57990) [0x7fbdf99fd990]
> (EE) 3: /usr/bin/Xvnc (0x55cd52c44000+0x958d0) [0x55cd52cd98d0]
> (EE) 4: /usr/bin/Xvnc (0x55cd52c44000+0xddf4d) [0x55cd52d21f4d]
> (EE) 5: /usr/bin/Xvnc (0x55cd52c44000+0x8abc0) [0x55cd52ccebc0]
> (EE) 6: /usr/bin/Xvnc (0x55cd52c44000+0xe6b08) [0x55cd52d2ab08]
> (EE) 7: /usr/bin/Xvnc (dix_main+0x4a0) [0x55cd52d8f940]
> (EE) 8: /lib64/libc.so.6 (0x7fbdf99a6000+0x40eec) [0x7fbdf99e6eec]
> (EE) 9: /lib64/libc.so.6 (__libc_start_main+0x87) [0x7fbdf99e6fb5]
> (EE) 10: /usr/bin/Xvnc (_start+0x23) [0x55cd52cbbf11]
> (EE) 
> (EE) Segmentation fault at address 0x10
> (EE) 
> Fatal server error:
> (EE) Caught signal 11 (Segmentation fault). Server aborting
> (EE)

Comment 7 Stefan Hundhammer 2024-02-15 14:42:19 UTC

Looks more like an X11 / Xvnc problem to me. We had something similar just recently.

-> X maintainers

Comment 8 Stefan Hundhammer 2024-02-15 14:46:42 UTC

The last 'ps' snapshot from memsample.zcat:

> ### free-0196-2024-02-15T14:16:13+01:00
>                total        used        free      shared  buff/cache   available
> Mem:       264000100     3267988   261282820      620768     1285020   260732112
> Swap:        2097312           0     2097312
> ### ps-0196-2024-02-15T14:16:13+01:00
>   PID TTY           VSZ      DRS      TRS      RSS     SIZE  PPID COMMAND
>  3719 ttyS1        5252     5139      112     1536      444  3718     /bin/dash /sbin/inst_setup yast
>  3788 ttyS1        6320     5355      964     2560      532  3719       /bin/bash /usr/sbin/yast2
>  3789 ttyS1        9892     8927      964     2560     1184  3788         /bin/bash /usr/lib/YaST2/startup/YaST2.First-Stage
>  4887 ttyS1       10032     9067      964     3072     1324  3789           /bin/bash /usr/lib/YaST2/startup/YaST2.call installation initial
>  5396 ttyS1      187348   184903     2444    56808    10376  4887             /usr/bin/Xvnc :0 -noreset -rfbauth /root/.vnc/passwd.yast -desktop Installation -geometry 1024x768 -dpi 96 -rfbport 5901 -fp /usr/share/fonts/misc/,/usr/share/fonts/uni/,/usr/share/fonts/truetype/
>  5397 ttyS1       10032     9067      964     2480     1324  4887             /bin/bash /usr/lib/YaST2/startup/YaST2.call installation initial
>  5399 ttyS1       32228    32223        4    20660    12272  5397               python3 -c import websockify.websocketproxy; websockify.websocketproxy.websockify_init() --web /usr/share/novnc 5801 localhost:5901
> 49022 ttyS1        5748     5720       27     1536      516  4887             sleep 1

Comment 9 Stefan Hundhammer 2024-02-15 14:56:56 UTC

"hexdump" is not quite the correct term here; it's really an old-style uuencoded  core dump, the type of thing that your was common to send binary files in e-mails in the late 1980s / early 1990s.

Comment 10 Stefan Dirsch 2024-02-15 16:26:21 UTC

Known issue. Closing as duplicate.

*** This bug has been marked as a duplicate of bug 1219311 ***