Re: [gentoo-user] Limiting amount of memory a program can use.

[Rich Freeman]

On Sun, Aug 28, 2022 at 11:32 AM Wols Lists <antlists@××××××××××××.uk> wrote:
>
> On 28/08/2022 15:21, Rich Freeman wrote:
> > Something I wish linux supported was discardable memory, for
> > caches/etc.  A program should be able to allocate memory while passing
> > a hint to the kernel saying that the memory is discardable.
>
> Linux DOES have that ...
>
> I'm not sure how to use it, but you can pass a flag to open(), which
> says "do not cache this file".

That isn't what I was proposing.  That has been around for a while.
However, it only impacts caching/buffering of open files, which is
kernel memory, and not memory held by a process itself.

You wouldn't want to limit kernel file caching of a torrent file,
since those files are pretty ideal candidates for caching (since the
client is likely to send data not long after receiving it).  There
isn't really any benefit to this either in this case as the kernel
already automatically drops file cache memory as needed under memory
pressure.

I was referring to application caching, which is internal storage used
by an application that may or may not have anything to do with open
files.  A browser cache is a good example of something like this
(though that extends to disk caching often and those files would also
get cached by the kernel).  The kernel can't simply drop this memory
when it needs memory since there is no way for it to tell which memory
an application owns is used for this purpose, and no way currently to
handle the resulting segfaults when the application goes to read that
memory.  Reading memory is a CPU instruction, not a system call, so it
would trigger an exception at the CPU level which must be handled.

I'm sure something could be engineered but it is more complex and I'm
guessing nobody has seen the need for it yet.  Ideally you'd also have
ways for the OS to hint how much memory applications ought to consume
in this way.  After all, you might have two applications running at
the same time, which both want to use lots of extra RAM for caching if
it is available, but neither has any way of knowing that this
situation exists and it is undesirable if all the memory goes to the
first application that asks, and also undesirable if after everything
is done grabbing memory if there is a bunch of it sitting around doing
nothing.

TL;DR: cache is not necessarily the same as kernel file cache.

-- 
Rich