KS2012: memcg/mm: Moving zcache towards the mainline

By Michael Kerrisk
September 17, 2012

2012 Kernel Summit

During the 2012 Kernel Summit memcg/mm minisummit, Konrad Wilk briefly introduced zcache as a prelude to considering how to bring zcache into the mainline kernel. Zcache reserves a portion of memory for compressed page storage. As memory gets tight and the system is about to swap, the kernel will instead start compressing pages. This defers having to transfer pages to much slower secondary storage or, optimally, completely avoids doing so, depending on the workload.

Zcache currently lives in the staging tree and Greg Kroah-Hartman was recently asked what was necessary to move it out. Konrad asked if anyone in the room had a chance to review the code, but it was clear that no one had within the last year or so. He said that there are a few users interested in developing features on top of zcache, but that developing against something in staging is high-risk.

Andrew Morton felt that staging is basically a graveyard as far as zcache is concerned and maybe the time has come to try and move it to the core VM and force the memory-management developers to review it. It may also be necessary to revisit whether the feature is even necessary and whether distributions will support it. However, apparently there are already users of zcache and KVM is interested if the sizing of the cache can be properly managed; thus, it should be possible to justify moving it to core VM.

There were concerns that compressed RAM features always look good on paper, but that there are too many workloads whose pages do not compress well; there were also questions around how to determine the optimal size of the compressed cache. Andrea Arcangeli was also concerned that the first pages to be evicted from memory are, by definition of the LRU page order, the ones that are least likely to be used in the future. These are the pages that should be going to secondary storage and more frequently used pages should be going to zcache. As it stands, zcache may fill up with no-longer-used pages and then the system continues to move used pages from and to the disk.

Konrad says he can address these concerns, but also tried to manage expectations, saying that there were some known problems with the current zcache implementation and people should be aware that there are circumstances where it could trigger the OOM killer.

Next: Dirty/writeback LRU

Index entries for this article
Kernel	Memory management/Swapping

KS2012: memcg/mm: Moving zcache towards the mainline

Posted Sep 17, 2012 23:26 UTC (Mon) by Jonno (subscriber, #49613) [Link] (4 responses)

> As it stands, zcache may fill up with no-longer-used pages and then the system continues to move used pages from and to the disk.

IANAKD (I am not a kernel developer), but wouldn't the obvious solution be to not swap out application memory directly, but instead swap application memory to zcache, and once zcache fills start moving the oldest remaining zcache content to disc in order to make room for more? Thus no-longer-used pages would end up compressed on disc, and the system would only move used pages to and from zcache.

KS2012: memcg/mm: Moving zcache towards the mainline

Posted Sep 18, 2012 0:33 UTC (Tue) by brendan_wright (guest, #7376) [Link] (3 responses)

> wouldn't the obvious solution be to... swap application memory to zcache, and once zcache fills start moving the oldest remaining zcache content to disc...?

IANAKD either, but it sounds feasible to me :-)

KS2012: memcg/mm: Moving zcache towards the mainline

Posted Sep 19, 2012 23:03 UTC (Wed) by djm1021 (guest, #31130) [Link] (2 responses)

This is exactly the direction that zcache2 (now under the staging/ramster) is moving. The hard part is that (1) two (or more) pageframes must be available to decompress the compressed pages, at a point in time where pageframes are alerady scarce, (2) moving those decompressed pages to the swap disk must be properly synchronized with swapping non-zcache pages to disk as there are some interesting race possibilities. A preliminary patch is already working, but poorly and will need more review and input from mm experts.

KS2012: memcg/mm: Moving zcache towards the mainline

Posted Sep 20, 2012 10:35 UTC (Thu) by Jonno (subscriber, #49613) [Link] (1 responses)

Why are pages decompressed as part of moving the page to disk, rather than as part of swapping in it to memory again? I can see no benefit of not storing the already compressed page on disk, deferring the decompression to when the content is actually needed (if ever).

KS2012: memcg/mm: Moving zcache towards the mainline

Posted Sep 20, 2012 16:09 UTC (Thu) by djm1021 (guest, #31130) [Link]

What you suggest is certainly possible (but anything is possible in software :-), however it would take fairly massive changes to the swap subsystem and/or userland-visible changes. Currently, the swap cache is an in-RAM mapping of some subset of pages on the swap disk and this is important for maintaining coherency between the (one or more user or kernel) threads accessing those pages and the I/O reading/writing the swap disk. If some pageframe-size slots on the swapdisk contain one or more compressed pages, this becomes a lot more complex.

In the end, such changes might be necessary or be the best alternative, but we are trying less invasive approaches first.

KS2012: memcg/mm: Moving zcache towards the mainline

Posted Sep 20, 2012 3:59 UTC (Thu) by nwatkins (guest, #61119) [Link] (1 responses)

I'm curious about the policy by which zcache chooses the pages to be compressed, or if the choice is largely random?

KS2012: memcg/mm: Moving zcache towards the mainline

Posted Sep 20, 2012 16:11 UTC (Thu) by djm1021 (guest, #31130) [Link]

Definitely not random. See Transcendent Memory in a Nutshell http://lwn.net/Articles/454795/ especially section 3.