Toward the unification of hugetlbfs

Posted May 22, 2024 18:14 UTC (Wed) by adobriyan (subscriber, #30858)
Parent article: Toward the unification of hugetlbfs

One thing I never understood about hugepages is why they are connected to a filesystem _at_ all.

Do you ask for 4 KiB pages by mounting stuff and mapping files there? No! You just mmap them.

Another thing I learned about using MAP_HUGE is that 2 hugepage VMAs next to each other don't merge by default.

Toward the unification of hugetlbfs

Posted May 22, 2024 19:07 UTC (Wed) by willy (subscriber, #9762) [Link]

If you look through the Linux history, you'll find we used to have other mechanisms. eg in 2002 we committed a patch to remove sys_alloc_hugepages() and sys_free_hugepages()

Toward the unification of hugetlbfs

Posted May 22, 2024 19:25 UTC (Wed) by flussence (guest, #85566) [Link] (2 responses)

It was “a product of its time”. Early 2.x kernels didn't have most of the autotuning we take for granted nowadays, and this was a way to have *something* to let people squeeze out an extra 5% from their hardware using a fairly obscure feature.

Nowadays we have THP and madvise(MADV_HUGEPAGE) and neither needs hugetlbfs so it's a lot less important than it once was, but it's userspace API and in active use so it'll probably stick around for a good while to come.

And I mildly disagree - more things ought to be exposed as filesystems. I think networking innards should've been, for one, but I doubt that mountain will budge in 2024.

Toward the unification of hugetlbfs

Posted May 22, 2024 21:45 UTC (Wed) by WolfWings (subscriber, #56790) [Link]

Honestly the madvice() path is by far the best way forward in almost all circumstances. The only 'gap' being sharing pages between processes (not threads) at this time unless/until the mshare() approach lands.

Toward the unification of hugetlbfs

Posted May 31, 2024 5:29 UTC (Fri) by chleroy (guest, #171626) [Link]

At the time being THP is very limited to hugepage sizes that can fit as a single entry in PMD or PUD.

For all other hugepage sizes there is no other way that hugetlbfs to use them.