replaced by new patch from ..
replaced by new patch from ..
Posted Sep 29, 2005 10:36 UTC (Thu) by nix (subscriber, #2304)In reply to: replaced by new patch from .. by hisdad
Parent article: Swap prefetching
The patch you're discussing is intended to stop processes that would dirty many pages from filling up memory with dirty pages far faster than the device can emit them, and only then blocking; instead it forces them to block sooner, before so much of memory is filled up by its dirtied pages.
This is orthogonal to the patch under discussion, which is arranging that when something *has* filled memory and then freed it all again, that useful stuff gets put back in there sooner rather than later: after all, memory can be filled by many things other than dirtied pages awaiting flushing (e.g., large ld(1) runs ;) )
Posted Sep 29, 2005 13:50 UTC (Thu)
by liljencrantz (guest, #28458)
[Link] (3 responses)
The writeout on dirty pages patch fixes filter-type programs that read/write out huge amounts of data, but never use the same data for more than a short period of time. This is somewhat related to the new instructions in the next-generation consoles that support writing out data without touching the caches, only that the kernel autodetects such uses, so the program doesn't have to explicitly tell the OS what data not to put in the cache. This patch should in theory work very well for backup programs, indexers, media players and many other types of cache killers.
The swap prefetching patch would also solve the issue of filter-type programs, though significantly less efficiently, since a filter program would first force the entire system to swap out and then slowly 'swap in' during half a minute or so. But swap prefetching also fixes a slightly different type of problem, namely when an application uses a huge amount of memory and then exits (or at least free()s the memory). This includes applications that do some rather complicated things during initialization, like OpenOffice, as well as memory-hungry 'one-shot' programs, like yum.
Posted Sep 29, 2005 21:08 UTC (Thu)
by nix (subscriber, #2304)
[Link] (2 responses)
But it handles the other half of the job. :)
Posted Sep 29, 2005 21:35 UTC (Thu)
by giraffedata (guest, #1954)
[Link] (1 responses)
Actually, as VM changes so frequently, I don't know just what the present cache replacement policy is; maybe it's already sweep resistant; my point is that early writing isn't about that.
Early writing makes it so all those page frames wasted with pages that will never be used again are at least clean, so when they do get reclaimed, the reclaimer doesn't have to wait. The wait for page laundry is shifted to the guy diryting pages away from the innocent bystander competing for real memory.
Posted Sep 29, 2005 23:24 UTC (Thu)
by farnz (subscriber, #17727)
[Link]
Posted Sep 29, 2005 20:15 UTC (Thu)
by hisdad (subscriber, #5375)
[Link]
I had mostly thought of swap in the case of large copies and not considered
It will be interesting to see what they are like in practice.
--John
These algorithms may be orthogonal in what they do, but the problems they solve have a strong overlap.replaced by new patch from ..
It doesn't fix reading-cache-killers: they'll still push other data out of the cache.replaced by new patch from ..
The early writing of dirty pages patch doesn't actually address, except in an incidental way, the problem of single-use pages, either read or write. If you read or write a gigabyte of virtual memory on a .5 gigabyte system, you will sweep physical memory with the patch just like without. To solve that problem, you need to change the cache replacement policy, not the prewriting policy.
replaced by new patch from ..
AIUI, the big gain of Andrea's early write-out patch is that the VM has a
strong preference for evicting clean pages (nearly free) over dirty
(expensive I/O needed). Because Andrea's code stops streaming writes like
cp from dirtying lots of one-use pages, it makes the pages that cp is
dirtying much more attractive to the VM when it's hunting for freeable
pages.
replaced by new patch from ..
Ah yes, angeli. so i didn't recall correctly afterall!replaced by new patch from ..
these other cases.