Bug 1224451 - Kernel update kernel-default-5.14.21-150500.55.59.1-x86_64 or higher break NFS mounts using krb5i/p
Summary: Kernel update kernel-default-5.14.21-150500.55.59.1-x86_64 or higher break NF...
Status: RESOLVED DUPLICATE of bug 1224340
Alias: None
Product: openSUSE Distribution
Classification: openSUSE
Component: Kernel:Filesystems (show other bugs)
Version: Leap 15.5
Hardware: x86-64 openSUSE Leap 15.5
: P5 - None : Major (vote)
Target Milestone: ---
Assignee: Kernel Filesystems
QA Contact: E-mail List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-05-18 11:19 UTC by J A Gow
Modified: 2024-05-18 17:26 UTC (History)
1 user (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description J A Gow 2024-05-18 11:19:04 UTC
Kernels 5.14.21-150500.55.59.1-x86_64 and greater break NFS mounting. This manifested at first on a system using autofs with user information stored in ldap and using NFS-mounted home directories. Kernels prior to 5.14.21-150500.55.59.1-x86_64 (5.14.21-150500.55.52.1-x86_65 or lower) work fine.

With no change in configuration, updating to 5.14.21-150500.55.59.1-x86_64 caused the clients to hang immediately prior to the display manager login page being displayed. The clients can be accessed remotely at this point via ssh, and dmesg shows the following exception:

[   34.139479] nfs4: Deprecated parameter 'intr'
[   34.146013] Key type dns_resolver registered
[   34.348189] NFS: Registering the id_resolver key type
[   34.348203] Key type id_resolver registered
[   34.348204] Key type id_legacy registered
[   34.443269] BUG: kernel NULL pointer dereference, address: 0000000000000004
[   34.443291] #PF: supervisor read access in kernel mode
[   34.443300] #PF: error_code(0x0000) - not-present page
[   34.443309] PGD 0 P4D 0 
[   34.443318] Oops: 0000 [#1] PREEMPT SMP NOPTI
[   34.443329] CPU: 8 PID: 4235 Comm: mount.nfs4 Tainted: P           OE  X  N 5.14.21-150500.55.62-default #1 SLE15-SP5 c363f6b34a951d50530113de3511a86499c762cd
[   34.443345] Hardware name: ASUSTeK COMPUTER INC. WS-C621E-SAGE Series/WS-C621E-SAGE Series, BIOS 6605 05/26/2021
[   34.443354] RIP: 0010:gss_get_mic_kerberos+0x2d/0x300 [rpcsec_gss_krb5]
[   34.443372] Code: 00 00 41 57 41 56 41 55 41 54 49 89 d5 55 53 49 89 f4 48 83 ec 38 48 8b 5f 08 65 48 8b 04 25 28 00 00 00 48 89 44 24 30 31 c0 <8b> 43 04 83 f8 06 0f 84 fa 00 00 00 0f 86 eb 00 00 00 83 e8 11 83
[   34.443386] RSP: 0018:ffffab4ccbfc7800 EFLAGS: 00010246
[   34.443396] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff971142b1cfd4
[   34.443405] RDX: ffffab4ccbfc7890 RSI: ffffab4ccbfc78b0 RDI: ffff9710cc2979c0
[   34.443414] RBP: ffffab4ccbfc7930 R08: ffff971142b1c008 R09: ffff971142b1cfd4
[   34.443422] R10: ffffab4ccbfc7948 R11: ffffdf35860ac700 R12: ffffab4ccbfc78b0
[   34.443430] R13: ffffab4ccbfc7890 R14: ffff9710c4a65140 R15: ffff971142b1c020
[   34.443438] FS:  00007f23bd037840(0000) GS:ffff971fffe00000(0000) knlGS:0000000000000000
[   34.443448] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   34.443457] CR2: 0000000000000004 CR3: 00000013f9bc6002 CR4: 00000000007706e0
[   34.443466] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   34.443474] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[   34.443483] PKRU: 55555554
[   34.443489] Call Trace:
[   34.443496]  <TASK>
[   34.443507]  gss_marshal+0x181/0x320 [auth_rpcgss 0f368b4c04963921938df1845be6981ef9b3b687]
[   34.443540]  ? kmalloc_order+0x6f/0xb0
[   34.443559]  ? __rpc_sleep_on_priority_timeout+0xf0/0xf0 [sunrpc 21bd2d2d2f7dd857ffba00711d9cf31d8a1e34b3]
[   34.443674]  ? call_encode+0x151/0x310 [sunrpc 21bd2d2d2f7dd857ffba00711d9cf31d8a1e34b3]
[   34.443767]  call_encode+0x151/0x310 [sunrpc 21bd2d2d2f7dd857ffba00711d9cf31d8a1e34b3]
[   34.443860]  ? rpc_malloc+0x4d/0x90 [sunrpc 21bd2d2d2f7dd857ffba00711d9cf31d8a1e34b3]
[   34.443963]  ? call_decode+0x210/0x210 [sunrpc 21bd2d2d2f7dd857ffba00711d9cf31d8a1e34b3]
[   34.444054]  __rpc_execute+0xb4/0x4d0 [sunrpc 21bd2d2d2f7dd857ffba00711d9cf31d8a1e34b3]
[   34.444156]  rpc_execute+0x6b/0xd0 [sunrpc 21bd2d2d2f7dd857ffba00711d9cf31d8a1e34b3]
[   34.444259]  rpc_run_task+0x111/0x130 [sunrpc 21bd2d2d2f7dd857ffba00711d9cf31d8a1e34b3]
[   34.444354]  nfs4_run_exchange_id+0x1fe/0x300 [nfsv4 d59cef2e53679b0000b74b5adfd2d9a4aadf9e3e]
[   34.444438]  ? select_task_rq_fair+0x186/0x13b0
[   34.444455]  _nfs4_proc_exchange_id+0x34/0x550 [nfsv4 d59cef2e53679b0000b74b5adfd2d9a4aadf9e3e]
[   34.444532]  ? try_to_wake_up+0x177/0x550
[   34.444544]  nfs4_proc_exchange_id+0x3e/0x50 [nfsv4 d59cef2e53679b0000b74b5adfd2d9a4aadf9e3e]
[   34.444626]  nfs41_discover_server_trunking+0x1c/0xa0 [nfsv4 d59cef2e53679b0000b74b5adfd2d9a4aadf9e3e]
[   34.444712]  nfs4_discover_server_trunking+0x6f/0x2a0 [nfsv4 d59cef2e53679b0000b74b5adfd2d9a4aadf9e3e]
[   34.444795]  nfs4_init_client+0x6e/0x130 [nfsv4 d59cef2e53679b0000b74b5adfd2d9a4aadf9e3e]
[   34.444882]  nfs4_set_client+0xef/0x140 [nfsv4 d59cef2e53679b0000b74b5adfd2d9a4aadf9e3e]
[   34.444967]  nfs4_create_server+0x128/0x2b0 [nfsv4 d59cef2e53679b0000b74b5adfd2d9a4aadf9e3e]
[   34.445053]  nfs4_try_get_tree+0x33/0xd0 [nfsv4 d59cef2e53679b0000b74b5adfd2d9a4aadf9e3e]
[   34.445135]  vfs_get_tree+0x22/0xd0
[   34.445148]  path_mount+0x6f2/0x9c0
[   34.445161]  do_mount+0x79/0x90
[   34.445173]  __x64_sys_mount+0x86/0xe0
[   34.445184]  do_syscall_64+0x58/0x80
[   34.445200]  ? exit_to_user_mode_prepare+0xfe/0x240
[   34.445214]  ? syscall_exit_to_user_mode+0x1f/0x40
[   34.445225]  ? do_syscall_64+0x67/0x80
[   34.445238]  ? syscall_exit_to_user_mode+0x1f/0x40
[   34.445248]  ? syscall_exit_to_user_mode+0x1f/0x40
[   34.445258]  ? do_syscall_64+0x67/0x80
[   34.445270]  ? do_syscall_64+0x67/0x80
[   34.445281]  ? do_user_addr_fault+0x1ff/0x730
[   34.445296]  ? syscall_exit_to_user_mode+0x1f/0x40
[   34.445305]  ? do_syscall_64+0x67/0x80
[   34.445317]  ? exc_page_fault+0x67/0x150
[   34.445327]  entry_SYSCALL_64_after_hwframe+0x6b/0xd5
[   34.445342] RIP: 0033:0x7f23bd33f30a
[   34.445352] Code: c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 66 90 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 56 1b 0d 00 f7 d8 64 89 01 48
[   34.445366] RSP: 002b:00007ffd27b707c8 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5
[   34.445378] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f23bd33f30a
[   34.445386] RDX: 000055f69bd26ba0 RSI: 000055f69bd26b80 RDI: 000055f69bd288f0
[   34.445395] RBP: 00007ffd27b70940 R08: 000055f69bd28ef0 R09: 0032362e312e3836
[   34.445403] R10: 0000000000000000 R11: 0000000000000246 R12: 000055f69bd28b30
[   34.445411] R13: 0000000000000010 R14: 00007ffd27b70850 R15: 000055f69bd28bc0
[   34.445422]  </TASK>
[   34.445428] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache netfs ppdev parport_pc parport vmw_vsock_vmci_transport vsock vmw_vmci af_packet iscsi_ibft iscsi_boot_sysfs qrtr(N) ns(N) cmac algif_hash algif_skcipher af_alg bnep dmi_sysfs binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common raid1 skx_edac nfit md_mod libnvdimm x86_pkg_temp_thermal intel_powerclamp coretemp nls_iso8859_1 nls_cp437 kvm_intel vfat fat kvm snd_hda_codec_realtek uvcvideo snd_hda_codec_generic snd_hda_codec_hdmi snd_usb_audio btusb ledtrig_audio iTCO_wdt btrtl btbcm videobuf2_vmalloc intel_pmc_bxt btintel iTCO_vendor_support videobuf2_memops btmtk videobuf2_v4l2 snd_usbmidi_lib videobuf2_common snd_hda_intel snd_rawmidi snd_intel_dspcfg joydev snd_intel_sdw_acpi bluetooth snd_seq_device pcspkr irqbypass snd_hda_codec igb snd_hda_core i2c_i801 ecdh_generic lpc_ich rfkill snd_hwdep i2c_smbus mfd_core crc16 i2c_algo_bit
[   34.445556]  mei_me ioatdma mei intel_pch_thermal dca acpi_pad auth_rpcgss sunrpc fuse configfs ip_tables x_tables xfs libcrc32c hid_logitech_hidpp hid_logitech_dj uas usb_storage hid_generic usbhid nvidia_drm(POEX) nvidia_modeset(POEX) nvidia_uvm(POEX) sr_mod cdrom sd_mod nvidia(POEX) crc32_pclmul crc32c_intel ghash_clmulni_intel mxm_wmi(N) drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops nvme drm aesni_intel ahci nvme_core libahci crypto_simd cryptd nvme_common xhci_pci libata xhci_pci_renesas t10_pi xhci_hcd crc64_rocksoft_generic crc64_rocksoft crc64 wmi button dm_mirror dm_region_hash dm_log vhci_hcd usbip_core usbcore v4l2loopback(OEN) videodev mc snd_aloop snd_pcm snd_timer snd soundcore sg dm_multipath dm_mod scsi_dh_rdac scsi_dh_emc scsi_dh_alua scsi_mod msr
[   34.445734] Supported: No, Proprietary and Unsupported modules are loaded
[   34.445742] CR2: 0000000000000004
[   34.445751] ---[ end trace e35e8d7dc70ac129 ]---
[   34.460894] RIP: 0010:gss_get_mic_kerberos+0x2d/0x300 [rpcsec_gss_krb5]
[   34.460906] Code: 00 00 41 57 41 56 41 55 41 54 49 89 d5 55 53 49 89 f4 48 83 ec 38 48 8b 5f 08 65 48 8b 04 25 28 00 00 00 48 89 44 24 30 31 c0 <8b> 43 04 83 f8 06 0f 84 fa 00 00 00 0f 86 eb 00 00 00 83 e8 11 83
[   34.460915] RSP: 0018:ffffab4ccbfc7800 EFLAGS: 00010246
[   34.460921] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff971142b1cfd4
[   34.460927] RDX: ffffab4ccbfc7890 RSI: ffffab4ccbfc78b0 RDI: ffff9710cc2979c0
[   34.460932] RBP: ffffab4ccbfc7930 R08: ffff971142b1c008 R09: ffff971142b1cfd4
[   34.460937] R10: ffffab4ccbfc7948 R11: ffffdf35860ac700 R12: ffffab4ccbfc78b0
[   34.460942] R13: ffffab4ccbfc7890 R14: ffff9710c4a65140 R15: ffff971142b1c020
[   34.460947] FS:  00007f23bd037840(0000) GS:ffff971fffe00000(0000) knlGS:0000000000000000
[   34.460954] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   34.460959] CR2: 0000000000000004 CR3: 00000013f9bc6002 CR4: 00000000007706e0
[   34.460964] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   34.460970] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[   34.460975] PKRU: 55555554
[   34.510796] nfs4: Deprecated parameter 'intr'

I have access to the NFS server (which is also openSuSE Leap 15.5 stock) - and the above exception is caused when the NFS server is running kernel-default-5.14.21-150500.55.52.1-x86_64.

A manual attempt to mount an NFS partition results in the same exception.

If the NFS server kernel is upgraded to 5.14.21-150500.55.59.1 or higher, the bug is still present insofar as it is impossible to mount an exported NFS directory. The exception is not generated and the mount just fails silently, resulting in the home directories not being mounted on the client at login, and manual mounts fail silently.

No change in configuration of server or client between changes of kernel. I can boot the older 55.52.1 kernel on both server and client and everything works as expected.

This is also confirmed across all the NFS clients in the network, each of which has different hardware, graphics etc. All clients and server work fine with 55.52.1, anything higher and NFS is broken. Same exception.
Comment 1 hui 2024-05-18 15:05:12 UTC
.

*** This bug has been marked as a duplicate of bug 1224340 ***