Re: [gentoo-amd64] Re: System becomes almost unusable when compling c++ code

[Volker Armin Hemmann]

On Freitag, 3. August 2007, Richard Freeman wrote:
> Volker Armin Hemmann wrote:
> > where is the difference between 'app syncs its files to disk' and 'kernel
> > swaps app to disk, than decides to swap it in, so app can decides which
> > files it wants to sync to disk, then swap it out again'?
> >
> > oh, more disk access...
>
> Actually, you're not quite looking at it the right way.  Suppose during
> the course of an emerge I need 150MB of temp files.  I have two choices
> - write to disk, or write to tmpfs.
>
> In the former case every write goes to disk buffer (ram), and then is
> synced to disk within about 5 seconds.  In the latter case every write
> goes to tmpfs (ram), and if necessary gets written to swap at an
> opportune time (when the system needs more ram, the page is idle for a
> long time, the system is otherwise idle, etc) - and if this need doesn't
> come up before the file is deleted it never gets written to disk at all.
>
> In the worst case both get synced to disk, but in the former case the
> kernel does not get to choose when outside the forced setting of about 5
> seconds.  In the latter case the kernel can schedule disk access more
> opportunistically - because the kernel knows the data isn't that
> important and doesn't need to be written to disk quickly.
>
> Generally well-written apps don't decide what files it wants syned to
> disk - the kernel does that.  If a file is written to a normal
> filesystem it gets synced according to the rules of the filesystem, and
> if it is written to tmpfs it gets written to disk if the system needs
> more ram and the file is more idle than most of the rest of RAM.
>
> I think you'd be surprised at how similarly files on disk and memory in
> ram get handled.  Both get "swapped" in and out of memory (if the file
> is mmapped - which is often the case), and writes to files get stored in
> buffers/cache just like files in tmpfs.  The big difference is that
> tmpfs files are not guaranteed to ever get written to disk, which means
> the kernel can handle them more flexibly.  Normal files are ALWAYS
> written to disk (unless they're deleted in the few seconds before they
> are synced), and the kernel isn't allowed to just keep them in buffers
> for 30 minutes if the disk is busy.
>
> In the extreme of excessive RAM the tmpfs is going to be many orders of
> magnitude faster than writing to disk.  In the extreme of a very
> ram-starved system it will probably be comparable.
>
> > And honestly? From my 'feelings' swap is way slower than normal disk
> > accesses.
>
> What kinds of access are you comparing?  If you're talking about
> swapping in and out a 50MB binary vs a 50kb text file then you're
> comparing apples to oranges.

I am comparing an open konqueror window that was minimized, had a webpage open 
and was swapped out with the rest of X and the KDE session compared with log 
in, start konqueror and loading that page.

You see, both give me a usable window in the same timeframe (+/- 1 or 2 
seconds) but the second choice does not make the mouse jerking around.

The worst: as soon as say 100mb are in swap, everything is slow for the rest 
of the evening. Best solution?
swapoff -a && swapon -a. And have some popcorn when the disk is accessed like 
mad.

>even if you have no swap partition.  Linux just mmaps the
> executable into memory and starts running it (with little to nothing in
> RAM) - any time the program accesses its own code it triggers a swap-in.
>
I know that - and it is faster to read that executable from fs than to swap it 
in....
-- 
gentoo-amd64@g.o mailing list