Re: [gentoo-user] Re: [SOLVED] =dev-lang/spidermonkey-60.5.2_p0-r4 fails @world update

[Michael]

On Saturday, 20 June 2020 04:18:24 BST urpion@×××.com wrote:
> On Sat, Jun 20, 2020 at 08:56:52AM +1000, urpion@×××.com wrote:
> > On Fri, Jun 19, 2020 at 06:08:49PM +0100, Michael wrote:

> > > A single compile job on a big package can eat >3G of RAM.  I don't know
> > > what spidermonkey jobs may grow up to, but drop it down to '-j1' and
> > > see what you get.
> > >
> > > A swapfile which you enable on a per ebuild basis if required, can be
> > > handy. You can configure this in a package.profile file. On an ext4 fs
> > > you could do this as root:
> > >
> > > touch swapfile
> > > dd if=/dev/zero of=swapfile bs=4096 count=400000
> > > mkswap -L Swappage swapfile
> > > swapon swapfile
> > >
> > > If it is a btrfs you'll need to tweak things to be able to used it as a
> > > swap:
> > >
> > > https://wiki.archlinux.org/index.php/Swap#Swap_file
> >
> > Wow. Thanks Micheal. That's really cool. I will try that.
>
> I tried to merge the package again with -j1, just to see, but no go.
> Making a swap file just as Micheal said did the trick. Thanks a bunch.
> I made it in /home. Is that a bad idea? I guess I need to review my
> partion scheme. Thanks everyuone, it's great to be back on Gentoo again
>
> (-: :-)

OK, you're definitely short on RAM then, but only for big monster compiles.  
I'd suggest you give Chromium a miss!  ;-)

Here's some general thoughts on this topic:

Big packages can consume a lot of RAM even for a single job.  A swap partition 
or file will help, but you could soon arrive in a temporary race condition if 
you use higher counts for MAKEOPTS jobs or if you're also using heavily the PC 
at the time, placing demands on memory.  If you have opened up a mail client, 
a dozen tabs on a browser, some music/video players, and are running a 
database & webserver, transcoding, etc. you'll find even a single job on a 
hungry compile will start paging in and out of swap continuously, thrashing 
the disk and applications/desktop will become unresponsive for minutes at a 
time.

Memory management then becomes necessary to avoid situations like this.

1. SWAP

You can add a swap partition, although this will require repartitioning.  
Alternatively, add a swapfile as I mentioned above.  It doesn't matter if you 
have the file under / or under /home, or some other data partition(s).  What 
matters is the file is owned by root:root and for security is set with 0600 
access rights.  If you're paranoid about security then the file can be placed 
on an encrypted partition.  I think I had set up a swapfile on a MSWindows 
NTFS partition once, because there was spare capacity there, so it pretty much 
does not matter where you set it up - although security may be a concern.

It is worth discussing the fs on which you place a swapfile.  As I have 
mentioned already BTRFS can be a problem, especially if you are running older 
kernels:

https://btrfs.wiki.kernel.org/index.php/FAQ#Does_btrfs_support_swap_files.3F

Defragging the fs before you create a swapfile may also be beneficial on 
spinning disks, to avoid fragmentation.  Other than these points, with modern 
kernels there shouldn't be much difference between a swapfile or a swap 
partition in performance terms.  If anything the swapfile has the benefit of 
being able to resize it if you need more space - or add a second swapfile too, 
on the same or different fs.

You don't have to use the swapfile every day, every time, if it only becomes 
necessary for large compiles.  In this case you don't need to add it in /etc/
fstab.  However, having some swap available at all times it won't hurt your 
system either.  The Linux kernel is intelligent enough to use it when needed.  
You could always interfere with swappiness to specify yourself how much and 
how often RAM is copied over to swap.  The default is 60:

$ sysctl vm.swappiness
vm.swappiness = 60

You can reduce it to swap less aggressively, or increase it to swap more so.


2. I/O BLOCK SCHEDULER

If at some point swapping starts (heavily) and you already have -j=1, you may 
discover the machine becomes less responsive.  Changing the kernel's block 
scheduler will help on spinning disks.

On a spinning disk the input/output queuing and buffering of data as is 
written and read off the disk can become a constraint, especially during heavy 
swapping, or heavy stints of writing on /var/tmp/portage, thereby slowing 
everything down to the point of frustration.  Changing the kernel block 
scheduler for a given disk or partition can help here.  The default is mq-
deadline:

$ cat /sys/class/block/sda/queue/scheduler
[mq-deadline] kyber bfq none

Changing this to bfq will improve responsiveness:

# echo bfq > cat /sys/class/block/sda/queue/scheduler

The above pointers ought to get you out of sticky situations when compiling 
large packages.

Happy emerging!  :-)