Re: [gentoo-user] Dual or Quad CPU complications?

[Rafa Griman]

On Fri, Dec 14, 2012 at 10:44 AM, Grant <emailgrant@×××××.com> wrote:
>> > So if I have 2 physical CPU's with 4 cores each and I enable SMP, I'm
>> > using
>> > 8 cores?  Can NUMA be either enabled or disabled when using more than
>> > one
>> > physical CPU, or is it required?
>>
>>
>> NUMA is a hardware architecture. It's how you access memory on a
>> hardware level: NUMA = Non Uniform Memory Access vs a UMA architecture
>> of typical (old/legacy) SMP systems (UMA = Uniform Memory Access).
>>
>> In a UMA system, all the memory belongs to all the sockets. In a NUMA
>> system, each socket has it's "own" local memory. In modern (x86-64)
>> processors, each socket has it's own memory controller so each socket
>> controls its own local memory. If one socket runs out of memory it can
>> ask another socket to lend him some memory. In a UMA system, no socket
>> has to ask since memory is global and belongs to all sockets so if one
>> socket uses up all the memory ... the rest "starve". In NUMA, there's
>> more control over who uses what (be it cores or RAM).
>>
>> If you have a modern dual or quad (or higher #) socket system ...
>> you've got NUMA architecture and you can't get rid of it, it's
>> hardware, not software.
>
> So I must enable CONFIG_NUMA for more than one physical CPU, and disable it
> for only one physical CPU?


Yup. But ... Why would you want to disable a socket (CPU)? If you
disable a socket (CPU) ... you lose the memory attached to that socket
(CPU) not to mention you lose those cores ;)

A better solution would be to use cgroups or numactl tools to pin a
certain process to a set of cores and a memory region.

If you really want to deactivate cores (but not the whole socket), you can type:

          echo 0 > /sys/devices/system/cpu/cpu1/online

This would deactivate core #1. You can deactivate as many cores as you
wish, except for core #0.

This can be done without rebooting your server (aka during run time).
Your memory will not be affected, but you will have less cores (and
theoretically more memory bandwidth). I say "theoretically" because
you always have to benchmark these things with YOUR application
(remember logic NEVER applies to real life ;)

If you want to check the # of cores you've got:

     cat /proc/interrupts | grep CPU

Other possibilities such as cat /proc/cpuinfo or dmesg, ... can be
useful too for this: your choice, FLOSS gives you options.

If you want to activate the previously deactivated core, you can run:

          echo 1 > /sys/devices/system/cpu/cpu1/online

Now ... be sure your core numbering is the expected core numbering.
IOW, not all server vendors follow the same numbering scheme so core
#1 in vendor A's server could be core #2 in vendor B's server. Never
trust logic ;)

As I mentioned previously: test/benchmark YOUR software. DON'T trust
logic or generic benchmarks or web pages with results. Trust YOUR
results only.

HTH

   Rafa