Bug 1211541

Kernel team, can you please confirm whether preempt=full is set on purpose? I'm highly confident that we've originally agreed on full.  Thank you!

### BEGIN /etc/grub.d/10_linux ###
menuentry 'openSUSE Leap 15.5'  --class opensuse --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-12758e3f-2d51-4a39-8220-13a44c835722' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_gpt
        insmod btrfs
        search --no-floppy --fs-uuid --set=root 12758e3f-2d51-4a39-8220-13a44c835722
        echo    'Loading Linux 5.14.21-150500.52-default ...'
        linux   /boot/vmlinuz-5.14.21-150500.52-default root=UUID=12758e3f-2d51-4a39-8220-13a44c835722  ${extra_cmdline} splash=silent resume=/dev/disk/by-uuid/e2c08f4c-5fdd-4f9c-96bf-efc9d03c2fee preempt=full mitigations=auto quiet security=apparmor
        echo    'Loading initial ramdisk ...'
        initrd  /boot/initrd-5.14.21-150500.52-default
}

I thought this behavior was same on Leap 15.4?  The setup comes from YaST, and the use of preempt=full is intentional, IIRC.  preempt=full is the upstream default, and it makes more sense for desktop usage.

(In reply to Takashi Iwai from comment #3)
> I thought this behavior was same on Leap 15.4?  The setup comes from YaST,
> and the use of preempt=full is intentional, IIRC.  preempt=full is the
> upstream default, and it makes more sense for desktop usage.

Hello Takashi,

I recall it differently: in Bug 1125004 "Change default kconfig: CONFIG_PREEMPT_VOLUNTARY" from 2019 we switched Tumbleweed and Leap 15.1 to "voluntary preemption". They used to be on "full preemption". The command line argument didn't exist at the time of that bug, but CONFIG_PREEMPT_VOLUNTARY translates to preempt=voluntary.

Regarding if it's best or not, the common wisdom (as you know) is that throughput with Full preemption can be penalized, and so Voluntary seems like a safer choice; if we want to go beyond the common wisdom and run tests, I'm 100% on board and indeed I've been tasked with it for the next few weeks, but that can't happen this week (deadline for the change, see Luboš on comment 1).

Takashi:
From Bug 1125004 it really sounds like we went through this and selected Voluntary. Do you recall that discussion?

I'm approaching this bug with the same spirit as Luboš: be it voluntary, be it full, there are merits for both, let's only make sure it's by deliberate choice and not by accident. My contribution here is to remind you all that in 2019 we switched full->voluntary, and I'd trust our past selves with that decision. (Bug 1125004).

Question for Luboš (needinfo):
In comment 1 you write: "I'm highly confident that we've originally agreed on full".
Can you confirm this statement? Full preemption, not Voluntary? It's odd, as if this was what you recall, you wouldn't have opened this bug, so I suspect a typo.

Michal Kubeček:
I'm CC'ing you as you were involved in the various discussions (above bug from 2019, then later in 2020) on the kernel preemption flavor to adopt for desktop. Luboš showed me we're about to go with preempt=full while I was quite sure we settled on Voluntary.

AFAICT this is successor to kernel-preempt which had full preemption in SP3, with kernel-default having none. Voluntary preemption was not provided.

With the dynamic preempt users can now choose voluntary preempt but that's not what we had before the dynamic preempt at all.

(In reply to Giovanni Gherdovich from comment #4)
> 
> Regarding if it's best or not, the common wisdom (as you know) is that [...]

Takashi, I feel the need to clarify why I wrote "as you know" in the sentence before, as it can be interpreted in several ways.
My intent was to honestly acknowledge your expertise on the matter.

IMO, the current setup (preempt=full) should be OK for Leap.  preempt=full must work well for the typical desktop usages.  If this is chosen as default for SLE-Server, it's a problem, though.

IIRC, the primary reason TW chose preempt=voluntary was a regression report on certain desktop usage (bsc#1112824).  Although the change of preempt mode wasn't confirmed to be the real culprit, it seemed that aligning the all preempt setups (there were inconsistencies at that time among architectures) could be a benefit, and it might be "safer" to align with what Fedora used.  That was discussed in bsc#1125004.

Meanwhile, SLE was with preempt=none, and the switch to preempt=voluntary didn't happen on SLE (thus Leap as well) at that time, because it was verified that preempt=voluntary caused a performance regression compared to preempt=none.  There was kernel-preempt package provided for the preempt=full, and this was supposed to be the use for the desktop.

On SLE15-SP3, we revisited the preempt setup, and implemented the dynamic change between preempt=none and preempt=voluntary for making kernel-default usable for desktops (jsc#SLE-16775).  In the kernel default, preempt=none was still kept, but can be switched to preempt=voluntary via boot option.

Then on SLE15-SP4, further unification of preempt=full happened (jsc#SLE-18857), and finally kernel-preempt package was gone and merged into kernel-default.  The kernel default itself was still preempt=none, but it can switch to both preempt=voluntary and preempt=full.

Later on, since 5.16, TW also took the dynamic preempt, too, but it keeps preempt=voluntary as the default.  Maybe because we didn't discuss further about the switch.

The above was all about the kernel kconfig.  The remaining question is the setup of preempt=xxx boot option (supposedly) by YaST, and I have no details.

Ok let's keep preempt=full on Leap.

For context, this present bug/question BOO#1211541 was opened by Luboš as last week I gave a presentation claiming that Leap uses "Voluntary Preemption" (I was ill-informed), then Luboš showed me it's on "Full Preemption", since he made the Yast change himself last year to Leap 15.4 following jsc#SLE-18857 "Further converge preempt kernel mode". So Luboš and I were confused and opened this bug to double check and make sure we're all on the same page.

Takashi in comment 3 and comment 7 confirms preempt=full is a fine choice for desktop. To corroborate that, note how we've been shipping Leap 15.4 with preempt=full for a year and there have been no reported issues.
"Full" used to be Leap's setting up to 2019 (CONFIG_PREEMPT), it was changed to Voluntary as described by Takashi but the motivation wasn't fully satisfactory (alleged Gnome performance regression, never fully demonstrated). With the implementation of dynamic preemption, in 2022 we reverted to Full with this Yast change https://github.com/yast/skelcd-control-openSUSE/pull/253
Full Preemption the mean recommended by upstream to enhance responsiveness/interactivity so let's use it. Historically (early 2000's) the Voluntary Preemption was intended to get at least some preemption in the hands of users, while Full Preemption had the reputation of being unstable. A stop-gap of sort. That was almost 20 years ago.

CC'ing Michal Hocko and Mel Gorman so we're all aware: Leap ships preempt=full by default.

I'll close this bug/question by the end of the week if there are no further comments.
Thanks Luboš for raising this and Takashi and others for the feedback.

Thank you for confirmation team! At least now we have a tracked justification.

Thanks Luboš for closing, I forgot to do that.

Adding one more piece of information for future reference: yesterday I asked Michal Hocko (author of jsc#SLE-18857 "Further converge preempt kernel mode") what was his expectation for preemption on Leap when he wrote the Jira ticket. He told me he totally expected to get Leap back to preempt=full, for all the reasons outlined in comment 7 by Takashi. So really the only one who didn't get the memo was me :)
Anyways, now I know.