[gentoo-user] Re: tmp on tmpfs

[Ian Zimmerman]

On 2017-05-24 08:00, Kai Krakow wrote:

> While I have no benchmarks and use the systemd default of tmpfs for
> /tmp, I also put /var/tmp/portage on tmpfs, automounted through
> systemd so it is cleaned up when no longer used (by unmounting).
>
> What can I say? It works so much faster: Building packages is a lot
> faster most of the time, even if you'd expect gcc uses a lot of
> memory.
>
> Well, why might that be? First, tmpfs is backed by swap space, that
> means, you need a swap partition of course. Swap is a lot simpler than
> file systems, so swapping out unused temporary files is fast and is a
> good thing. Also, unused memory sitting around may be swapped out
> early. Why would you want inactive memory resident? So this is also a
> good thing. Portage can use memory much more efficient by this.
>
> Applying this reasoning over to /tmp should no explain why it works so
> well and why you may want it.
>
> BTW: I also use zswap, so tmpfs sits in front of a compressed
> write-back cache before being written out to swap compressed. This
> should generally be much more efficient (performance-wise) than
> putting /tmp on zram.
>
> I configured tmpfs for portage to use up to 30GB of space, which is
> almost twice the RAM I have. And it works because tmpfs is not
> required to be resident all the time: Inactive parts will be swapped
> out. The kernel handles this much similar to the page cache, with the
> difference that your files aren't backed by your normal file system
> but by swap.  And swap has a lot lower IO overhead.
>
> Overall, having less IO overhead (and less head movement for portage
> builds) is a very very efficient thing to do. GCC constantly needs all
> sorts of files from your installation (libs for linking, header files,
> etc), and writes a lot of transient files which are needed once later
> and then discarded. There's no point in putting it on a non-transient
> file system.
>
> I use the following measures to get more performance out of this
> setup:
>
>   * I have three swap partitions spread across three HDDs
>   * I have a lot of swap space (60 GB) to have space for tmpfs
>   * I have bcache in front of my HDD filesystem
>   * I have a relatively big SSD dedicated to bcache
>
> My best recommendation is to separate swap and filesystem devices.
> While I didn't do it that way, I still separate them through bcache
> and thus decouple fs access and swap access although they are on the
> same physical devices. My bcache is big enough that most accesses
> would go to the SSD only. I enabled write-back to have that effect
> also for write access.
>
> If you cannot physically split swap from fs, a tmpfs setup for portage
> may not be recommended (except you have a lot of memory, like 16GB or
> above). But YMMV.
>
> Still, I recommend it for /tmp, especially if your system is on SSD.

All interesting points, and you convinced me to at least give tmpfs a
try on the desktop.

My laptop is different, though.  It doesn't have that much RAM by
comparison (4G) and it _only_ has a SSD.  Builds have been slow :(  I am
afraid to mess with it lest I increase the wear on the SSD.

> Unix semantics suggest that /tmp is not expected to survive reboots
> anyways (in contrast, /var/tmp is expected to survive reboots), so
> tmpfs is a logical consequence to use for /tmp.

/tmp is wiped by the bootmisc init job anyway.

-- 
Please *no* private Cc: on mailing lists and newsgroups
Personal signed mail: please _encrypt_ and sign
Don't clear-text sign:
http://primate.net/~itz/blog/the-problem-with-gpg-signatures.html