Bugzilla – Bug 1212775
go1.16 fails to build with new update kernel-sources 6.4
Last modified: 2024-06-25 17:44:39 UTC
go1.16 fails to build when updating kernel-sources to 6.4, see: https://build.opensuse.org/package/live_build_log/openSUSE:Factory:Staging:M/go1.16/standard/i586 https://build.opensuse.org/package/live_build_log/openSUSE:Factory:Staging:M/go1.16/standard/x86_64 In both cases, it's after the step "Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1." For i586 things go bad after this: [ 409s] strconv [ 409s] releasep: m=0x579e2000 m->p=0x5781c600 p->m=0x0 p->status=2 [ 409s] fatal error: releasep: invalid p state [ 409s] While on x86_64, there is an out of memory: [ 325s] hash/adler32 [ 325s] hash/crc32 [ 325s] cmd/internal/codesign [ 336s] fatal error: runtime: out of memory
*** Bug 1212774 has been marked as a duplicate of this bug. ***
Note: it happens only 1-3 times per 20 cases. So it needs care during bisection. Bisecting...
I bisected to: commit 0bff0aaea03e2a3ed6bfa302155cca8a432a1829 (refs/bisect/bad) Author: Suren Baghdasaryan <surenb@google.com> Date: Mon Feb 27 09:36:28 2023 -0800 x86/mm: try VMA lock-based page fault handling first Reverting the commit on the top of 6.4 seems to fix the issue.
Reported upstream at: https://lore.kernel.org/all/20230227173632.3292573-30-surenb@google.com/T/#m606a7a3168a277106a726a17176d3bf189f8ba10
Jiri, Do you think this should to be reported to Go upstream at this point? i.e. this commit is slated to remain in kernel 6.4? There have been previous intermittent invalid p state failures in the Go compiler: go#15246 runtime: fatal error: releasep: invalid p state (2016) https://github.com/golang/go/issues/15246
(In reply to Jeff Kowalczyk from comment #5) > Jiri, Do you think this should to be reported to Go upstream at this point? Can't hurt to make them aware. > i.e. this commit is slated to remain in kernel 6.4? It's part of a large series that would be painful to revert. I can imagine if finding a fix is not successful and more things start failing, the functionality could be disabled in 6.4.x. > There have been previous intermittent invalid p state failures in the Go > compiler: > > go#15246 runtime: fatal error: releasep: invalid p state (2016) > https://github.com/golang/go/issues/15246 Hm so that means it's possible go e.g. always did something racy, but it was rare to manifest with mmap_lock, but it can be more likely exposed with the newly possible parallelism thanks to per-vma locking? Nasty.
(In reply to Vlastimil Babka from comment #6) > > go#15246 runtime: fatal error: releasep: invalid p state (2016) > > https://github.com/golang/go/issues/15246 > > Hm so that means it's possible go e.g. always did something racy, but it was > rare to manifest with mmap_lock, but it can be more likely exposed with the > newly possible parallelism thanks to per-vma locking? Nasty. Yeah, it makes sense to talk to go guys too. OTOH, the issue is from 2016, have we seen failures since then?
Do newer Go versions build with 6.4? Currently go1.x bootstraps with an earlier go1.x toolchain, but gccgo is an option for bootstrapping. This change is already under consideration primarily to shorten the bootstrap chain. go1.16 and go1.17 are EOL for upstream updates: https://github.com/golang/go/wiki/Go-Release-Cycle The supported versions today are go1.19 and go1.20, and in August 2023 the supported versions shift to go1.20 and go1.21 (go1.21rc2 is in Factory). TBD if 6.4 compatibility fixes for recent Go versions backport to go1.16 and go1.17. go1.17 in particular is a designated bootstrap version: https://github.com/golang/go/issues/44505
(In reply to Jeff Kowalczyk from comment #8) > Do newer Go versions build with 6.4? Only both go1.16 and go1.17 were failing. > Currently go1.x bootstraps with an earlier go1.x toolchain, but gccgo is an > option for bootstrapping. This change is already under consideration > primarily to shorten the bootstrap chain. > > go1.16 and go1.17 are EOL for upstream updates: > https://github.com/golang/go/wiki/Go-Release-Cycle The supported versions > today are go1.19 and go1.20, and in August 2023 the supported versions shift > to go1.20 and go1.21 (go1.21rc2 is in Factory). > > TBD if 6.4 compatibility fixes for recent Go versions backport to go1.16 and > go1.17. go1.17 in particular is a designated bootstrap version: > https://github.com/golang/go/issues/44505 It looks like a bug in these old versions. So the kernel doesn't seem to be at fault this time. So could you somehow eliminate these old unsupported versions of go? (I actually quite don't understand why they designated an unsupported version for bootstrapping?)
There is a minimal reproducer in the kernel.org but that does not require go.
It was fixed in 6.5-rc1 by commits: > fb49c455 fork: lock VMAs of the parent process when forking > 2b4f3b49 fork: lock VMAs of the parent process when forking > 1c7873e3 mm: lock newly mapped VMA with corrected ordering > 33313a74 mm: lock newly mapped VMA which can be modified after it becomes visible > c137381f mm: lock a vma before stack expansion Pushed to stable. (master is already at rc1)