[gentoo-amd64] Re: Kernel scheduler seems to be making mistakes

[Duncan]

Peter Humphrey <prh@××××××××××.uk> posted
200705201511.52195.prh@××××××××××.uk, excerpted below, on  Sun, 20 May
2007 15:11:52 +0100:

> This is turning into a bad month  :-(

They happen. =8^(

> I'm running BOINC clients on this box, and the kernel seems unable to
> schedule them properly. I'm subscribed to several projects, so I should
> have one on each CPU all the time, running at nice 19 and therefore
> mopping up all available CPU cycles. That's how it used to run. But
> nowadays the kernel scheduler insists on allocating both of them to the
> same CPU, thus limiting them to 50% load.

I noticed the same here -- two or more CPU heavy programs often seem to 
get stuck on the same CPU now, leaving the other one running virtually 
idle. =8^(  I figured it was dynamics related to the fact that pretty 
much all the I/O and peripherals are hooked up to the one (CPU 0), making 
it the more efficient one to run stuff like X on, or anything doing a lot 
of I/O, while the other is kind of hanging out alone, with only memory 
and the other CPU directly attached.  (Note that it's a dual Opteron 
system, not a dual core, thus it's not two cores in the same socket with 
everything attached to it, but two different sockets, with just about 
everything attached to socket 0, probably so it's easier to run a single-
CPU and leave the other empty.)

While it might be possible for someone who really knows what they are 
doing to tweak things to get more even distribution automatically, I 
decided fiddling with that was way over my head, so I elected to go the 
other route.

The route I took, you have two choices for tools, both in the sys-process 
category.  schedtool compiles into a single binary executable that you 
call with different parameters depending on what you want it to do.  
schedutils takes the opposite approach, with several tools, each of which 
do a particular thing.  I chose schedutils since I was mainly interested 
in only the one tool from it anyway, and learning it while ignoring the 
others I think is easier than learning the complexities of a single big 
complex tool most of the functions of which I'm not particularly 
interested in.

So, if you emerge schedutils, one of the binaries you get is called 
taskset.  Once it's emerged, you can read the notes on taskset in /usr/
share/doc/schedutils-*/README.bz2, and/or the taskset manpage.  It's 
pretty simple, to use, however.  For example, on a two-core or two-CPU 
system (CPU0 and CPU1), setting an already running X to run on CPU0 only, 
on CPU1 only, or on both, is done with the following commands (the number 
being a CPU bitmask, obviously):

taskset -p 1 `pidof X`
taskset -p 2 `pidof X`
taskset -p 3 `pidof X`

The binary bitmasks of course are
01=1
10=2
11=3

Any bit that's a one is a CPU the process can run on.  So in hex 0xFF, in 
decimal 255, in binary 1111 1111, would let a process run on any of the 
first 8 CPUs/cores.

It DOES caution not to set it to run on NO CPUs (0 up there in place of 
the number).  If it was a minor user process, it would probably simply 
suspend (I've not tried it to see), but try it with something like X when 
you are IN X, and things could get a bit "interesting".)

To start a new process while assigning it a specific CPU, you can do this:

taskset 2 tail -f /var/log/messages

That of course sets it to run on CPU1 (as opposed to CPU0).

There's another way of mapping them too, for those who find it hard to 
think in binary, but I've not used it so don't remember it.  It's not 
hard to memorize the binary bitmaps for 4 CPUs/cores anyway, 0xF in hex, 
and even easier to memorize it for only two CPUs, as I have now.

> It's also odd that CPU1 runs 5 - 6 C hotter than CPU0, whether loaded or
> not.

That's likely simply a calibration or sensor placement error in one or 
the other.  Here, CPU1 always runs /cooler/ than CPU0, but I think it's 
because the sensor for CPU1 isn't quite in such direct contact as is the 
CPU0 sensor, as the CPU0 sensor is always more sensitive as well. CPU0 
almost seems as affected by case temp as CPU utilization, so for all I 
know it's not touching the CPU at all.  (As the cost to do so was little 
more than the cost of the bare parts, bought from the pricewatch.com low 
price leaders separately, I bought the mobo with the CPUs and fans 
already mounted and pre-tested working, and have never had them off, so 
I've never seen /where/ the sensor is actually placed.)

As long as your CPUs will shut down before they burn up, and they should, 
I'd not worry about it.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@g.o mailing list