1 |
> >> > So if I have 2 physical CPU's with 4 cores each and I enable SMP, I'm |
2 |
> >> > using |
3 |
> >> > 8 cores? Can NUMA be either enabled or disabled when using more than |
4 |
> >> > one |
5 |
> >> > physical CPU, or is it required? |
6 |
> >> |
7 |
> >> |
8 |
> >> NUMA is a hardware architecture. It's how you access memory on a |
9 |
> >> hardware level: NUMA = Non Uniform Memory Access vs a UMA architecture |
10 |
> >> of typical (old/legacy) SMP systems (UMA = Uniform Memory Access). |
11 |
> >> |
12 |
> >> In a UMA system, all the memory belongs to all the sockets. In a NUMA |
13 |
> >> system, each socket has it's "own" local memory. In modern (x86-64) |
14 |
> >> processors, each socket has it's own memory controller so each socket |
15 |
> >> controls its own local memory. If one socket runs out of memory it can |
16 |
> >> ask another socket to lend him some memory. In a UMA system, no socket |
17 |
> >> has to ask since memory is global and belongs to all sockets so if one |
18 |
> >> socket uses up all the memory ... the rest "starve". In NUMA, there's |
19 |
> >> more control over who uses what (be it cores or RAM). |
20 |
> >> |
21 |
> >> If you have a modern dual or quad (or higher #) socket system ... |
22 |
> >> you've got NUMA architecture and you can't get rid of it, it's |
23 |
> >> hardware, not software. |
24 |
> > |
25 |
> > So I must enable CONFIG_NUMA for more than one physical CPU, and |
26 |
disable it |
27 |
> > for only one physical CPU? |
28 |
> |
29 |
> |
30 |
> Yup. But ... Why would you want to disable a socket (CPU)? If you |
31 |
> disable a socket (CPU) ... you lose the memory attached to that socket |
32 |
> (CPU) not to mention you lose those cores ;) |
33 |
|
34 |
Sure but it sounds like if my system only has one CPU socket, CONFIG_NUMA |
35 |
should be disabled. |
36 |
|
37 |
- Grant |
38 |
|
39 |
|
40 |
> A better solution would be to use cgroups or numactl tools to pin a |
41 |
> certain process to a set of cores and a memory region. |
42 |
> |
43 |
> If you really want to deactivate cores (but not the whole socket), you |
44 |
can type: |
45 |
> |
46 |
> echo 0 > /sys/devices/system/cpu/cpu1/online |
47 |
> |
48 |
> This would deactivate core #1. You can deactivate as many cores as you |
49 |
> wish, except for core #0. |
50 |
> |
51 |
> This can be done without rebooting your server (aka during run time). |
52 |
> Your memory will not be affected, but you will have less cores (and |
53 |
> theoretically more memory bandwidth). I say "theoretically" because |
54 |
> you always have to benchmark these things with YOUR application |
55 |
> (remember logic NEVER applies to real life ;) |
56 |
> |
57 |
> If you want to check the # of cores you've got: |
58 |
> |
59 |
> cat /proc/interrupts | grep CPU |
60 |
> |
61 |
> Other possibilities such as cat /proc/cpuinfo or dmesg, ... can be |
62 |
> useful too for this: your choice, FLOSS gives you options. |
63 |
> |
64 |
> If you want to activate the previously deactivated core, you can run: |
65 |
> |
66 |
> echo 1 > /sys/devices/system/cpu/cpu1/online |
67 |
> |
68 |
> Now ... be sure your core numbering is the expected core numbering. |
69 |
> IOW, not all server vendors follow the same numbering scheme so core |
70 |
> #1 in vendor A's server could be core #2 in vendor B's server. Never |
71 |
> trust logic ;) |
72 |
> |
73 |
> As I mentioned previously: test/benchmark YOUR software. DON'T trust |
74 |
> logic or generic benchmarks or web pages with results. Trust YOUR |
75 |
> results only. |
76 |
> |
77 |
> HTH |
78 |
> |
79 |
> Rafa |