[gentoo-amd64] Re: System becomes almost unusable when compling c++ code

[Duncan]

Shaochun Wang <scwang@××××××.cn> posted 20070803065913.GA23254@localhost,
excerpted below, on  Fri, 03 Aug 2007 14:59:13 +0800:

> Every time I compile C++ code, e.g. app-i18n/scim-qtimm, my desktop
> system becomes almost not
> interactive. I have already set PORTAGE_NICENESS="15" in /etc/make.conf.
>
> Any suggestion?

Do you have a single-core single-CPU system, or multi-one-or-the-other?

In any case, unless you are running folding@home or the like, something 
truly idle-only that you want the emerge to get higher priority than, you 
should consider PORTAGE_NICENESS=19.   The reason being, a +19 nice is 
treated as idle priority by the scheduler, giving the rest of the system 
slightly higher responsiveness (lower latency), while giving the idle 
task somewhat longer time slices.  The effect can actually be to /speed/ 
/up/ compiles over a positive niceness <19 or even over normal 
scheduling, due to the longer timeslices.

What clock tick setting are you using, and what is your preemption 
setting?  Particularly if you are single-core and CPU, a higher clock 
tick setting (Timer frequency), not 100 certainly (that's for servers), 
probably 300 or 1000, will increase responsiveness at the cost of tasks 
taking longer (shorter timeslices, more overhead processing timeslices 
each second).  Similarly with preemption.  You'll want that set to 
Preemptible Kernel (Low-Latence Desktop) or at least Voluntary Kernel 
Premption (Desktop).  Also be sure the Preempt the Big Kernel Lock option 
is toggled ON.

Conversely, lower settings, No Forced Preemption (Server) or Voluntary 
Kernel Preemption (Desktop), and 100, 250 or 300 tick rate should work 
better for multi-core or multi-CPU SMP systems, because they can spread 
the load a bit more.  FWIW, dual Opteron 242 (so dual single cores) here, 
I'm running voluntary preemption, BKL preemption, and 300 Hz tick 
frequency.  That's the highest I'd recommend for a general purpose multi-
core or multi-CPU system, but tho I'd recommend as above, 1000 Hz tick 
and full preemption for single CPU/core systems.

You may also wish to play with the MAKEOPTS setting, typically -jX, where 
X is the number of CPUs/cores plus one to 150% of the CPUs/cores.  Thus, 
a single core/cpu system's recommended setting is -j2.  However, you may 
find -j1 increases your responsiveness.  Or you can try -j2, but add -l1 
or the like.  With GNU make (but not all others, you may have to remove 
the -lX portion for some merges), that'll tell it to allow up to two jobs 
(the -j2) but ONLY start a second one if the load average is below 1 (-
l1).  That's generally fairly effective.

There are other things to consider as well.  How do you usually compile, 
in a terminal window (xterm, konsole, gterm, etc) or at the text 
console?  I've noted that at least with konsole and with composite 
rendering (real transparency) turned on, non-niced CPU usage goes thru 
the roof trying to keep the konsole updated at times.  Causing the 
konsole window not to display, either minimizing it, shading it, or 
switching to a different desktop workspace so the konsole isn't shown, 
eliminates the issue.  If I'm /really/ planning on going to town (say a 
new KDE release came out and I have 100 plus packages of mostly C++ to 
compile), I'll turn off composite rendering as well.  When there's rapid 
display updates such as when compile output is scrolling by, if X and the 
X clients don't have to do all that extra work drawing and compositing 
areas normally hidden by other windows, it makes a big difference.  (It 
should be noted that typical composite overhead is <5% of a single CPU, 
here, more like 2% unless I've a huge bunch of windows open, and that's 
on a dual 1600x1200 display.  Radeon 9250, the last Radeons for which 
there's decent freedomware drivers, FWIW, tho the reverse engineering 
effort on the r300 and r400 series is progressing nicely.)

Then there's the standard stuff, but it'd tend to affect more than simply 
C++ compiling.  Make sure your SATA/PATA/SCSI chipsets are running their 
correct drivers with DMA enabled, not just generic, no DMA compatibility 
mode.  That's a big one but if it affected you you'd probably notice it 
elsewhere as well.  

If you have lots of memory (2 gigs or better, 4 gigs is nice, I have 8 
but that's overkill), strongly consider setting up your PORTAGE_TMPDIR 
(/var/tmp by default) on tmpfs.  Having all those temporary files 
typically used during a compile and merge written to memory only, instead 
of having to wait for several orders of magnitude slower hard disk 
access, DRAMATICALLY speeds up compiles, while at the SAME time speeding 
up general system responsiveness during the merge, because disk access 
slows down the /entire/ system, especially when whatever else you are 
working on is trying to access the disk at the same time.

To give you an example of what things /can/ be like, with my now aging 
dual Opteron 242 setup here (soon to be upgraded to dual-core Opteron 
290s), with 8 gigs memory (as I said, overkill, 4 gig would be fine), /
tmp on tmpfs (with $PORTAGE_TMPDIR=/tmp), 4-disk RAID-6 system (two 
parity stripes, so effectively 2-way striped), RAID-0 $PORTDIR and 
ccache, I routinely run MAKEOPTS="-j1000" (not that it ever gets there, 
but some builds don't like the unlimited -j, no number), and run five or 
more parallel emerges at the same time (using emerge -pt to get a tree 
output and -a for verification, so I can setup non-conflicting parallel 
emerges).  That's usually for a KDE update where I have 100 or so C++ KDE 
packages to merge, so it's mostly C++.  Because the config and certain 
other sections aren't parallelized and thus do only a single job, even 
with that, my load average seldom rises above 20 or 25.  Or, on the 
kernel, which is C not C++ but parallelizes VERY well, I'll get a load 
average of several /hundred/.  It's fun to see it go that high! =8^)  
Still, even with a 300-500 load average or 20ish load average on C++ (the 
most I seem to hit is 30), while I do get a bit of lag on the mouse, and 
the panel clock and ksysguard displays sometimes freeze for 10 seconds at 
a time, it's still surprising to me it's not /entirely/ unusable.  As 
well, I can be playing an Internet radio stream the entire time (and no, 
I don't have anything set real-time, either), with few if any dropouts at 
all.  That's REALLY astounding to me!  Up to a 500 load average, yet the 
scheduler continues to work well enough to give the network, player and 
audio system all the time it needs to prevent both dropped network 
packets and dropped audio data!  (It's obvious the scheduler prioritizes 
both the IP stack and the audio system without intervention, and equally 
obvious the KDE panel doesn't get the same auto prioritization, as 
clearly, 10 second updates as on the panel simply wouldn't cut it on the 
network or audio stream.  That be as it may, it's still fascinating to 
watch, seeing the load average climb to several hundred when I 
deliberately compile a kernel with -j1000, without a single hitch or skip 
in the audio playback at all!)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@g.o mailing list