Bugzilla – Bug 1213620
kernel oops: Unable to handle kernel paging request at virtual address 0000000000001ea0
Last modified: 2024-06-25 17:51:25 UTC
On openqa-aarch64 machine (aarch64 worker for openqa.opensuse.org) running Leap 15.5, we get lots of kernel oops related to qemu-system-aarch64 running 32-bit payload. This makes qemu tests to hang (originally reported at https://progress.opensuse.org/issues/132749) [ 249.997051] Unable to handle kernel paging request at virtual address 0000000000001ea0 [ 250.005675] Mem abort info: [ 250.009153] ESR = 0x0000000096000004 [ 250.013585] EC = 0x25: DABT (current EL), IL = 32 bits [ 250.019581] SET = 0, FnV = 0 [ 250.023318] EA = 0, S1PTW = 0 [ 250.027142] FSC = 0x04: level 0 translation fault [ 250.032702] Data abort info: [ 250.036268] ISV = 0, ISS = 0x00000004 [ 250.040788] CM = 0, WnR = 0 [ 250.044439] user pgtable: 4k pages, 48-bit VAs, pgdp=000000109ff43000 [ 250.051564] [0000000000001ea0] pgd=0000000000000000, p4d=0000000000000000 [ 250.059044] Internal error: Oops: 96000004 [#1] PREEMPT SMP [ 250.065305] Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace sunrpc fscache netfs af_packet nft_masq nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nfnetlink_cttimeout nft_chain_nat nf_tables ebtable_nat ebtable_broute ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat iptable_mangle iptable_raw tun iptable_security iscsi_ibft iscsi_boot_sysfs rfkill openvswitch nsh nf_conncount nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ipmi_ssif marvell ext4 mbcache nls_iso8859_1 jbd2 nls_cp437 vfat fat joydev acpi_ipmi ses efi_pstore(N) enclosure ipmi_si ipmi_devintf ipmi_msghandler button hns_dsaf hns_enet_drv hns_mdio hnae fuse ip_tables x_tables hid_generic usbhid sd_mod t10_pi crc64_rocksoft_generic crc64_rocksoft crc64 hibmc_drm drm_vram_helper drm_kms_helper syscopyarea [ 250.065445] ehci_platform sysfillrect crct10dif_ce sysimgblt fb_sys_fops ehci_hcd hisi_sas_v2_hw drm_ttm_helper ghash_ce ttm hisi_sas_main gf128mul libsas sha2_ce sha256_arm64 scsi_transport_sas drm sha1_ce usbcore sbsa_gwdt libata i2c_algo_bit usb_common i2c_designware_platform i2c_designware_core btrfs blake2b_generic libcrc32c xor xor_neon raid6_pq sg br_netfilter bridge stp llc dm_multipath dm_mod scsi_dh_rdac scsi_dh_emc scsi_dh_alua scsi_mod efivarfs aes_ce_blk crypto_simd cryptd aes_ce_cipher [ 250.196692] Supported: No, Unsupported modules are loaded [ 250.202780] CPU: 59 PID: 5913 Comm: qemu-system-aar Tainted: G N 5.14.21-150500.55.7-default #1 SLE15-SP5 b13f205f9f235d08d9f964520aed75460765e615 [ 250.218068] Hardware name: Huawei TaiShan 2280 /BC11SPCD, BIOS 1.50 06/01/2018 [ 250.225976] pstate: 600000c5 (nZCv daIF -PAN -UAO -TCO BTYPE=--) [ 250.232669] pc : kvm_arch_timer_get_input_level+0x20/0x80 [ 250.238768] lr : kvm_arch_timer_get_input_level+0x20/0x80 [ 250.244851] sp : ffff80001e35bb30 [ 250.248848] x29: ffff80001e35bb30 x28: 0000000000000000 x27: ffff800010ee9a30 [ 250.256668] x26: ffff8000120e5870 x25: ffff80001149c000 x24: 0000000000000000 [ 250.264487] x23: 0000000000000020 x22: 0000000000000000 x21: ffff0090e47fc7a0 [ 250.272308] x20: 000000000000001b x19: 000000000000001b x18: 0000000000000000 [ 250.280128] x17: 0000000000000000 x16: 0000000000000000 x15: 0000fffe7999cc20 [ 250.287947] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000 [ 250.295768] x11: 0000000000000000 x10: 0000000000000000 x9 : ffff8000100ea5a0 [ 250.303586] x8 : 0000000000000200 x7 : ffff009033ca1fc0 x6 : ffff009033ca1fc0 [ 250.311406] x5 : ffff80001e35bc50 x4 : 0000000000000000 x3 : ffff0090e47fe340 [ 250.319232] x2 : 0000000100000001 x1 : ffff009033ca1fc0 x0 : 0000000000000000 [ 250.327074] Call trace: [ 250.330208] kvm_arch_timer_get_input_level+0x20/0x80 [ 250.335951] vgic_get_phys_line_level+0x4c/0xc0 [ 250.341167] vgic_mmio_read_pending+0x128/0x1c0 [ 250.346383] vgic_uaccess+0x74/0x180 [ 250.350644] vgic_v2_dist_uaccess+0x64/0xc0 [ 250.355510] vgic_v2_attr_regs_access.isra.8+0x13c/0x180 [ 250.361509] vgic_v2_get_attr+0xa4/0x140 [ 250.366115] kvm_device_ioctl_attr+0xa0/0x100 [ 250.371167] kvm_device_ioctl+0xc0/0x100 [ 250.375793] __arm64_sys_ioctl+0xb8/0x100 [ 250.380500] invoke_syscall+0x74/0x100 [ 250.384938] el0_svc_common.constprop.4+0xa4/0x1c0 [ 250.390414] do_el0_svc+0x2c/0xc0 [ 250.394413] el0_svc+0x24/0x40 [ 250.398156] el0t_64_sync_handler+0x94/0xc0 [ 250.403022] el0t_64_sync+0x198/0x19c [ 250.407375] Code: 910003fd f9000bf3 2a0003f3 97ff5ec9 (b95ea001) [ 250.414155] ---[ end trace 29642f5e9d4313dd ]--- [ 250.419455] note: qemu-system-aar[5913] exited with preempt_count 1
This also happens on openqa worker `ip-10-252-32-90` which is an aws graviton machine running Leap 15.5: * https://openqa.opensuse.org/tests/3450732 * https://openqa.opensuse.org/tests/3450735 The problem arises while the qemu migration (snapshotting).
It looks like this [1] commit is addressing this crash. 15.4 will need same fix from what I see. [1] efedd01de475e ("KVM: arm64: Warn if accessing timer pending state outside of vcpu context")
Hi Guillaume, could you try kernel package from here, please? I do expect some warnings, but no crashes. [1] https://build.opensuse.org/project/show/home:iivanov:kernel-15.5-bsc1213620
Created attachment 868574 [details] kernel.log I installed the kernel from your repo, and the problem has been downgraded to a warning, indeed.
Hi, any chances that this could be tested on TW?
(In reply to Guillaume GARDET from comment #4) > Created attachment 868574 [details] > kernel.log > > I installed the kernel from your repo, and the problem has been downgraded > to a warning, indeed. Please note that the user space hang is still present, though. (In reply to Ivan Ivanov from comment #5) > Hi, any chances that this could be tested on TW? You mean try to reproduce with Tumbleweed kernel?
(In reply to Guillaume GARDET from comment #6) > (In reply to Guillaume GARDET from comment #4) > > Created attachment 868574 [details] > > kernel.log > > > > I installed the kernel from your repo, and the problem has been downgraded > > to a warning, indeed. > > Please note that the user space hang is still present, though. Ok. I believe package from here [1] should fix the issue completely. > > > (In reply to Ivan Ivanov from comment #5) > > Hi, any chances that this could be tested on TW? > > You mean try to reproduce with Tumbleweed kernel? Yes, but nevermind. I think I found what is missing [2]. I had already queued it for next kernel-update. Meanwhile, if you like you can test this one [1] once it is build. [1] https://build.opensuse.org/project/show/home:iivanov:kernel-15.5-bsc1213620_v2 [2] 2cdea19a34c2 ("KVM: arm64: Don't read a HW interrupt pending state ..")
(In reply to Ivan Ivanov from comment #7) > (In reply to Guillaume GARDET from comment #6) > > (In reply to Guillaume GARDET from comment #4) > > > Created attachment 868574 [details] > > > kernel.log > > > > > > I installed the kernel from your repo, and the problem has been downgraded > > > to a warning, indeed. > > > > Please note that the user space hang is still present, though. > > Ok. I believe package from here [1] should fix the issue completely. > > > > > > > (In reply to Ivan Ivanov from comment #5) > > > Hi, any chances that this could be tested on TW? > > > > You mean try to reproduce with Tumbleweed kernel? > > Yes, but nevermind. I think I found what is missing [2]. I had already > queued it for next kernel-update. Meanwhile, if you like you can test > this one [1] once it is build. > > [1] > https://build.opensuse.org/project/show/home:iivanov:kernel-15.5- > bsc1213620_v2 > [2] 2cdea19a34c2 ("KVM: arm64: Don't read a HW interrupt pending state ..") Yes, this package fixes the issue. At least, I was not able to reproduce so far.
(In reply to Guillaume GARDET from comment #8) > Yes, this package fixes the issue. At least, I was not able to reproduce so > far. Thank you! I think this is fixed.
SUSE-SU-2023:3302-1: An update that solves 28 vulnerabilities, contains two features and has 115 fixes can now be installed. Category: security (important) Bug References: 1150305, 1187829, 1193629, 1194869, 1206418, 1207129, 1207894, 1207948, 1208788, 1210335, 1210565, 1210584, 1210627, 1210780, 1210825, 1210853, 1211014, 1211131, 1211243, 1211738, 1211811, 1211867, 1212051, 1212256, 1212265, 1212301, 1212445, 1212456, 1212502, 1212525, 1212603, 1212604, 1212685, 1212766, 1212835, 1212838, 1212842, 1212846, 1212848, 1212861, 1212869, 1212892, 1212901, 1212905, 1212961, 1213010, 1213011, 1213012, 1213013, 1213014, 1213015, 1213016, 1213017, 1213018, 1213019, 1213020, 1213021, 1213024, 1213025, 1213032, 1213034, 1213035, 1213036, 1213037, 1213038, 1213039, 1213040, 1213041, 1213059, 1213061, 1213087, 1213088, 1213089, 1213090, 1213092, 1213093, 1213094, 1213095, 1213096, 1213098, 1213099, 1213100, 1213102, 1213103, 1213104, 1213105, 1213106, 1213107, 1213108, 1213109, 1213110, 1213111, 1213112, 1213113, 1213114, 1213116, 1213134, 1213167, 1213205, 1213206, 1213226, 1213233, 1213245, 1213247, 1213252, 1213258, 1213259, 1213263, 1213264, 1213272, 1213286, 1213287, 1213304, 1213417, 1213493, 1213523, 1213524, 1213533, 1213543, 1213578, 1213585, 1213586, 1213588, 1213601, 1213620, 1213632, 1213653, 1213705, 1213713, 1213715, 1213747, 1213756, 1213759, 1213777, 1213810, 1213812, 1213856, 1213857, 1213863, 1213867, 1213870, 1213871, 1213872 CVE References: CVE-2022-40982, CVE-2023-0459, CVE-2023-1829, CVE-2023-20569, CVE-2023-20593, CVE-2023-21400, CVE-2023-2156, CVE-2023-2166, CVE-2023-2430, CVE-2023-2985, CVE-2023-3090, CVE-2023-31083, CVE-2023-3111, CVE-2023-3117, CVE-2023-31248, CVE-2023-3212, CVE-2023-3268, CVE-2023-3389, CVE-2023-3390, CVE-2023-35001, CVE-2023-3567, CVE-2023-3609, CVE-2023-3611, CVE-2023-3776, CVE-2023-3812, CVE-2023-38409, CVE-2023-3863, CVE-2023-4004 Jira References: PED-4718, PED-4758 Sources used: openSUSE Leap 15.5 (src): kernel-livepatch-SLE15-SP5-RT_Update_3-1-150500.11.5.1, kernel-syms-rt-5.14.21-150500.13.11.1, kernel-source-rt-5.14.21-150500.13.11.1 SUSE Linux Enterprise Live Patching 15-SP5 (src): kernel-livepatch-SLE15-SP5-RT_Update_3-1-150500.11.5.1 SUSE Real Time Module 15-SP5 (src): kernel-syms-rt-5.14.21-150500.13.11.1, kernel-source-rt-5.14.21-150500.13.11.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
SUSE-SU-2023:3313-1: An update that solves 13 vulnerabilities and has 20 fixes can now be installed. Category: security (important) Bug References: 1206418, 1207129, 1210627, 1210780, 1211131, 1211738, 1212502, 1212604, 1212901, 1213167, 1213272, 1213287, 1213304, 1213585, 1213586, 1213588, 1213620, 1213653, 1213713, 1213715, 1213747, 1213756, 1213759, 1213777, 1213810, 1213812, 1213842, 1213856, 1213857, 1213863, 1213867, 1213870, 1213871 CVE References: CVE-2022-40982, CVE-2023-0459, CVE-2023-20569, CVE-2023-21400, CVE-2023-2156, CVE-2023-2166, CVE-2023-31083, CVE-2023-3268, CVE-2023-3567, CVE-2023-3609, CVE-2023-3611, CVE-2023-3776, CVE-2023-4004 Sources used: SUSE Linux Enterprise Micro for Rancher 5.4 (src): kernel-default-base-5.14.21-150400.24.81.1.150400.24.35.3 SUSE Linux Enterprise Micro 5.4 (src): kernel-default-base-5.14.21-150400.24.81.1.150400.24.35.3 Basesystem Module 15-SP4 (src): kernel-default-base-5.14.21-150400.24.81.1.150400.24.35.3, kernel-source-5.14.21-150400.24.81.1 Development Tools Module 15-SP4 (src): kernel-source-5.14.21-150400.24.81.1, kernel-syms-5.14.21-150400.24.81.1, kernel-obs-build-5.14.21-150400.24.81.1 SUSE Linux Enterprise Live Patching 15-SP4 (src): kernel-livepatch-SLE15-SP4_Update_16-1-150400.9.3.3 openSUSE Leap 15.4 (src): kernel-obs-qa-5.14.21-150400.24.81.1, kernel-default-base-5.14.21-150400.24.81.1.150400.24.35.3, kernel-syms-5.14.21-150400.24.81.1, kernel-source-5.14.21-150400.24.81.1, kernel-obs-build-5.14.21-150400.24.81.1, kernel-livepatch-SLE15-SP4_Update_16-1-150400.9.3.3 openSUSE Leap Micro 5.3 (src): kernel-default-base-5.14.21-150400.24.81.1.150400.24.35.3 openSUSE Leap Micro 5.4 (src): kernel-default-base-5.14.21-150400.24.81.1.150400.24.35.3 SUSE Linux Enterprise Micro for Rancher 5.3 (src): kernel-default-base-5.14.21-150400.24.81.1.150400.24.35.3 SUSE Linux Enterprise Micro 5.3 (src): kernel-default-base-5.14.21-150400.24.81.1.150400.24.35.3 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
SUSE-SU-2023:3311-1: An update that solves 15 vulnerabilities and has 27 fixes can now be installed. Category: security (important) Bug References: 1206418, 1207129, 1207948, 1210627, 1210780, 1210825, 1211131, 1211738, 1211811, 1212445, 1212502, 1212604, 1212766, 1212901, 1213167, 1213272, 1213287, 1213304, 1213417, 1213578, 1213585, 1213586, 1213588, 1213601, 1213620, 1213632, 1213653, 1213713, 1213715, 1213747, 1213756, 1213759, 1213777, 1213810, 1213812, 1213856, 1213857, 1213863, 1213867, 1213870, 1213871, 1213872 CVE References: CVE-2022-40982, CVE-2023-0459, CVE-2023-20569, CVE-2023-21400, CVE-2023-2156, CVE-2023-2166, CVE-2023-31083, CVE-2023-3268, CVE-2023-3567, CVE-2023-3609, CVE-2023-3611, CVE-2023-3776, CVE-2023-38409, CVE-2023-3863, CVE-2023-4004 Sources used: openSUSE Leap 15.5 (src): kernel-syms-5.14.21-150500.55.19.1, kernel-default-base-5.14.21-150500.55.19.1.150500.6.6.4, kernel-livepatch-SLE15-SP5_Update_3-1-150500.11.3.4, kernel-source-5.14.21-150500.55.19.1, kernel-obs-qa-5.14.21-150500.55.19.1, kernel-obs-build-5.14.21-150500.55.19.1 Basesystem Module 15-SP5 (src): kernel-default-base-5.14.21-150500.55.19.1.150500.6.6.4, kernel-source-5.14.21-150500.55.19.1 Development Tools Module 15-SP5 (src): kernel-obs-build-5.14.21-150500.55.19.1, kernel-syms-5.14.21-150500.55.19.1, kernel-source-5.14.21-150500.55.19.1 SUSE Linux Enterprise Live Patching 15-SP5 (src): kernel-livepatch-SLE15-SP5_Update_3-1-150500.11.3.4 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
SUSE-SU-2023:3318-1: An update that solves 20 vulnerabilities and has 89 fixes can now be installed. Category: security (important) Bug References: 1150305, 1193629, 1194869, 1206418, 1207129, 1207894, 1208788, 1210565, 1210584, 1210627, 1210780, 1210853, 1211131, 1211243, 1211738, 1211811, 1211867, 1212301, 1212502, 1212604, 1212846, 1212901, 1212905, 1213010, 1213011, 1213012, 1213013, 1213014, 1213015, 1213016, 1213017, 1213018, 1213019, 1213020, 1213021, 1213024, 1213025, 1213032, 1213034, 1213035, 1213036, 1213037, 1213038, 1213039, 1213040, 1213041, 1213059, 1213061, 1213087, 1213088, 1213089, 1213090, 1213092, 1213093, 1213094, 1213095, 1213096, 1213098, 1213099, 1213100, 1213102, 1213103, 1213104, 1213105, 1213106, 1213107, 1213108, 1213109, 1213110, 1213111, 1213112, 1213113, 1213114, 1213134, 1213167, 1213245, 1213247, 1213252, 1213258, 1213259, 1213263, 1213264, 1213272, 1213286, 1213287, 1213304, 1213523, 1213524, 1213543, 1213585, 1213586, 1213588, 1213620, 1213653, 1213705, 1213713, 1213715, 1213747, 1213756, 1213759, 1213777, 1213810, 1213812, 1213856, 1213857, 1213863, 1213867, 1213870, 1213871 CVE References: CVE-2022-40982, CVE-2023-0459, CVE-2023-20569, CVE-2023-20593, CVE-2023-21400, CVE-2023-2156, CVE-2023-2166, CVE-2023-2985, CVE-2023-31083, CVE-2023-3117, CVE-2023-31248, CVE-2023-3268, CVE-2023-3390, CVE-2023-35001, CVE-2023-3567, CVE-2023-3609, CVE-2023-3611, CVE-2023-3776, CVE-2023-3812, CVE-2023-4004 Sources used: openSUSE Leap 15.4 (src): kernel-syms-rt-5.14.21-150400.15.46.1, kernel-source-rt-5.14.21-150400.15.46.1 SUSE Linux Enterprise Live Patching 15-SP4 (src): kernel-livepatch-SLE15-SP4-RT_Update_11-1-150400.1.5.1 SUSE Real Time Module 15-SP4 (src): kernel-syms-rt-5.14.21-150400.15.46.1, kernel-source-rt-5.14.21-150400.15.46.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
SUSE-SU-2023:3376-1: An update that solves 15 vulnerabilities and has 27 fixes can now be installed. Category: security (important) Bug References: 1206418, 1207129, 1207948, 1210627, 1210780, 1210825, 1211131, 1211738, 1211811, 1212445, 1212502, 1212604, 1212766, 1212901, 1213167, 1213272, 1213287, 1213304, 1213417, 1213578, 1213585, 1213586, 1213588, 1213601, 1213620, 1213632, 1213653, 1213713, 1213715, 1213747, 1213756, 1213759, 1213777, 1213810, 1213812, 1213856, 1213857, 1213863, 1213867, 1213870, 1213871, 1213872 CVE References: CVE-2022-40982, CVE-2023-0459, CVE-2023-20569, CVE-2023-21400, CVE-2023-2156, CVE-2023-2166, CVE-2023-31083, CVE-2023-3268, CVE-2023-3567, CVE-2023-3609, CVE-2023-3611, CVE-2023-3776, CVE-2023-38409, CVE-2023-3863, CVE-2023-4004 Sources used: openSUSE Leap 15.5 (src): kernel-syms-azure-5.14.21-150500.33.14.1, kernel-source-azure-5.14.21-150500.33.14.1 Public Cloud Module 15-SP5 (src): kernel-syms-azure-5.14.21-150500.33.14.1, kernel-source-azure-5.14.21-150500.33.14.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
SUSE-SU-2023:3377-1: An update that solves 11 vulnerabilities and has 19 fixes can now be installed. Category: security (important) Bug References: 1206418, 1207129, 1210627, 1210780, 1211131, 1211738, 1212502, 1212604, 1212901, 1213167, 1213272, 1213287, 1213304, 1213588, 1213620, 1213653, 1213713, 1213715, 1213747, 1213756, 1213759, 1213777, 1213810, 1213812, 1213856, 1213857, 1213863, 1213867, 1213870, 1213871 CVE References: CVE-2022-40982, CVE-2023-0459, CVE-2023-20569, CVE-2023-21400, CVE-2023-2156, CVE-2023-2166, CVE-2023-31083, CVE-2023-3268, CVE-2023-3567, CVE-2023-3776, CVE-2023-4004 Sources used: openSUSE Leap 15.4 (src): kernel-source-azure-5.14.21-150400.14.63.1, kernel-syms-azure-5.14.21-150400.14.63.1 Public Cloud Module 15-SP4 (src): kernel-source-azure-5.14.21-150400.14.63.1, kernel-syms-azure-5.14.21-150400.14.63.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.