Re: [gentoo-user] mysqld invoked oom-killer

[Alan McKinnon]

On Thursday 21 July 2011 09:39:52 Grant did opine thusly:
 
> > My personal rule of thumb: if you hit swap, the bad thing has
> > already gone very very south, usually to the point where you
> > can't do much about it and it's already too late. Besides, that
> > bastard deomon spawn of satan called the oom-killer is likely
> > about to kick in and REALLY make your day. Anyone else notice
> > how oom-killer seems to be hard coded to zap the most
> > inconvenient process of all?.....
> >
> > What you need to be doing is monitor your memory usage during
> > normal conditions and deal with issues before they become
> > problems.
>
> Hi Alan, I think it was your advice I took a long time ago when I
> stopped installing new machines with a swap partition and disabled
> it on my already-installed machines.  Some time later, others on
> this list caught wind of what I'd done and told me I was an idiot.
> Is there a consensus on this?  If the drawbacks and advantages of
> using swap cancel each other out, I won't use it.

I would strongly advise you to make your own measurements and heed 
your own counsel. I can only speak from my own experience, and I may 
well be speaking a whole load of codswallop. Or I may be right and the 
opposing view is wrong. Who's to tell?

My own experience with backing swap has been almost uniformly bad, 
especially on machines running Apache and MySQL due to the massive 
performance hit it invariably causes. I see memory as a finite 
resource - you only have so much of it, so use it wisely and stay away 
from using all of it up.

The oom killer is also a point of contention. The algorithm is 
designed to try and detect the best pid to kill in order to keep the 
machine up, but there is no measurement for "least important process". 
So instead it has to infer it from time last used, time running and 
various other bits. These assumptions can never be 100% right.

I believe the best solution to running out of memory is well-written 
apps that degrade or die gracefully when they hit out of memory 
conditions. I always thought Apache was rather good at this, it would 
simply kill of the child process and deliver an error. You seem to 
have found a way round this :-)

kashani's advice seems reasonable, tune your machine to suit it's 
load. We've established that the problem was a client hitting your 
webserver 300 times in a minute. That is a DOS, so the solution would 
be to find a way to configure Apache to detect abuse like this and not 
deliver the page.

But back to swap. I have two cases where it is rather useful. This 
notebook uses swap for image storage when hibernating, and my Sybase 
database servers at work use swap well, retrieving data pages from 
swap is faster than searching through the database indexes for where 
they are, simply because the kernel alredy knows exactly where the 
swapped data is on disk. But this is a niche circumstance and in no 
way representative of a typical Linux machine's behaviour.

Summary: Do your own tests, make your own conclusions and vigorously 
defend them.

Sorry for the complete lack of a definitive answer, we are victims of 
TheRealWorldOutThere(tm)


-- 
alan dot mckinnon at gmail dot com