Re: [gentoo-user] Memory manager

[Dale]

Mick wrote:
> On Sunday, 20 October 2019 16:03:42 BST Dale wrote:
>> Here's the
>> thing about using swap on my rig, once it does, the system gets
>> extremely slow.  Even switching desktops can take a minute or longer.
>> Other than trying to get to what is eating up memory and killing it, the
>> system is virtually useless.  Even my video stops playing.
> Swapping can bring the system to its knees, but only under certain operating
> scenarios.  This is how I understand it works:
>
> Say you're browsing and keep opening tabs.  The browser application will
> preemptively allocate memory for more tabs, in case you carry on opening even
> more tabs.  Then you open yet another big application in terms of memory usage
> and start running it.  The kernel will reallocate some of the browser memory
> not currently utilised to the other application and keep things working
> smoothly.  With more applications/tabs being opened you will eventually run
> out of RAM and the kernel will swap some of the memory pages to disk.  The
> swapping is meant to be selective, i.e. things you haven't used in a while
> will be taken out of RAM and saved onto your disk.
>
> Under the above scenario you may notice a momentary latency on your desktop as
> data is swapped onto the disk, but afterwards the desktop should be responsive
> once more - unless more swapping is again demanded by your actions.  If you
> try to access an application which has had parts of its memory allocation
> swapped out to disk you will notice a delay in its reactions.
>
> Now, in a gentoo scenario, say a mammoth compile like Chromium, with a large
> count of jobs specified for it, you could end up swapping part or all of one
> or more jobs into memory, only to swap it out again in order to process it.
> The compile keeps swapping in and out a job at a time in order to carry on
> compiling.  The disk thrashing is now continuous and indeed interacting with
> your desktop will be painful - potentially waiting for minutes at a time
> before an application responds.  The way out of this bottleneck is to either
> increase your RAM, or minimise the use of memory by reducing the job count in
> MAKEOPTS.  Shutting down desktop applications and login out of any desktop
> sessions to release RAM will also help.
>
> On a laptop with 4G RAM compiling Chromium is quite challenging when even a
> single gcc job could grow to 3G or more.  Swapping and a disk I/O bottleneck
> becomes unavoidable and moving the compile of binaries to a bigger PC becomes
> a rather wise solution.
>
> Another occasion when swapping can cause havoc is when you have a memory leak
> due to some buggy application and all your RAM followed by swap is chewed up
> until an OOM ensues.
>
> For these reasons I always set up swap on my gentoo systems.


What you describe in your first scenario, that is when it is so slow and
virtually unresponsive.  I don't generally run out of memory when
compiling since I close other stuff, like Firefox, to free up memory,
even in the 16GB days.  What I run into is a tab, or tabs, in Firefox
that are chewing through memory like a hungry junk yard dog on a crook. 
When Firefox does that, it is slow, extremely slow.  When I get to where
I can see what is going on, it is still slow to even close or kill that
process.  If I get to a Konsole, I usually run htop and kill it from
there.  Thing is, it may take a minute even for htop to start and show
the problem.  That's a fairly small program but it can take a minute or
two to even load and show its screen.

I have two sites in particular that does this.  They run for days with
no problem and I may not even be using that tab or that site.  Then it's
like it gets mad and starts using more and more memory.  I have caught
it in time to just refresh the tab and it go back to normal.  Once it
starts using swap tho, it's very slow. 

I wish Firefox had a way to fix that out of control memory usage.  It
may not be Firefox itself doing it but it would seem it can see
something isn't right and put a stop to it.  It's sad when a Linux
desktop has 32GBs of memory eat up like that, usually by one program at
that. 

Still, no swap at all would result in a crash or reset.  It's better
than nothing but I wish the root cause could be fixed. 

Dale

:-)  :-)