Bug 1219284

Created attachment 872258 [details]
measures with kernel 6.7.1

Created attachment 872262 [details]
screenshot of free command 1

Created attachment 872263 [details]
screenshot of free command 2

Created attachment 872264 [details]
screenshot of free command 2

Any significant differences in /proc/slabinfo?

Created attachment 872295 [details]
/proc/slabinfo content on kernel 6.6

Created attachment 872296 [details]
/proc/slabinfo content on kernel 6.7

from a first glance I can't tell any significant difference, apart from user_namespace which is like doubled for 6.7 ; but some values like inode_cache or dentry are higher in 6.6 than 6.7 .

I attached the content of both measurement if you want to take a look.

$ grep user_namespace slabinfo_6.*

slabinfo_6.6.txt:user_namespace        13     13    624   13    2 : tunables    0    0    0 : slabdata      1      1      0

slabinfo_6.7.txt:user_namespace        25     25    632   25    4 : tunables    0    0    0 : slabdata      1      1      0

It seems, the Arch folks have done some debugging:

  https://bbs.archlinux.org/viewtopic.php?id=292086&p=3

and from a quick look this here might help:

  transparent_hugepage=madvise

If that's the case, let's invite MM people to the tea party.

6.6
AnonPages:        492536 kB
AnonHugePages:    264192 kB

6.7
AnonPages:        770116 kB
AnonHugePages:    399360 kB

There is much more anonymous memory used in 6.7 kernel (~270MB). Daniel has already mentioned that this might be THP related but only part of it can be contributed to that (135MB). So some userspace had to consume more memory with the newer kernel which is rather unexpected. Are you sure you are comparing the exactly same userspace here?

all the measurements were taken from the same fresh installed tumbleweed virtual machine with XFCE desktop environment, so same userspace. Just did these action to reproduce:

- boot choosing the 6.6 kernel
- start the DE 
- open a terminal to run commands 
- reboot
- choose 6.7 kernel
- start the DE 
- open a terminal to run commands

apparently the fix is:

c4608d1bf7c653 ("mm: mmap: map MAP_STACK to VM_NOHUGEPAGE")

Ah Thorsten Leemhuise corrected himself, it seems to be

  4ef9ad19e17676 ("mm: huge_memory: don't force huge page alignment on 32 bit")

but according some testing by the Arch folks it is not a complete fix.

(In reply to Daniel Wagner from comment #14)
> Ah Thorsten Leemhuise corrected himself, it seems to be
> 
>   4ef9ad19e17676 ("mm: huge_memory: don't force huge page alignment on 32
> bit")
> 
> but according some testing by the Arch folks it is not a complete fix.

I can see how this can over consume on thread's stacks because those are usually 2MB in size by default (by glibs) and the alignment would make it more likely THP target. I do not see THP would consume all the additional memory, though. Runtime overhead is likely a more likely visible effect.

That being said. No objection to taking the patch from my end.

(In reply to Daniel Wagner from comment #14)
> Ah Thorsten Leemhuise corrected himself, it seems to be
> 
>   4ef9ad19e17676 ("mm: huge_memory: don't force huge page alignment on 32
> bit")
> 
> but according some testing by the Arch folks it is not a complete fix.

Nope, this only affects 32b apps on 64b systems. And it mostly breaks ASLR (see bug 1218800. This should be already in stable kernel branch

an update, seems the problem has been found and fixed:

https://fosstodon.org/@kernellogger/111957388343037784

(In reply to Andrea Manzini from comment #17)
> an update, seems the problem has been found and fixed:
> 
> https://fosstodon.org/@kernellogger/111957388343037784

c4608d1bf7c6 ("mm: mmap: map MAP_STACK to VM_NOHUGEPAGE") should be part of our TW for about the week.

Is the problem still reproducible? I am mostly asking because I have some reservations this is the whole story as per comment 11

Created attachment 873086 [details]
free command output with kernel 6.7.6

did a quick test updating the same system to latest kernel 6.7.6 (screenshot attached); I can confirm the reported memory usage is lower.

Are we good to close this one?

closing this as fixed