1 |
On 19/10/14 04:15, James wrote: |
2 |
> thegeezer <thegeezer <at> thegeezer.net> writes: |
3 |
> |
4 |
> |
5 |
>> there is a little more here |
6 |
>> http://gentoo-en.vfose.ru/wiki/Improve_responsiveness_with_cgroups |
7 |
>> which will allow you to script creating a cgroup with the processID of |
8 |
>> an interactive shell, that you can start from to help save hunting down |
9 |
>> all the threads spawned by chrome. |
10 |
>> you can then do fun stuff with echo $$ > |
11 |
>> /sys/fs/cgroup/cpu/high_priority/tasks |
12 |
> Yea this is cool. But when it's a cluster, with thousands of processes |
13 |
cgroups are hierarchical, so for example if you start a bash script |
14 |
which is in cgroup "cpu/high_prio" which then starts your processes, all |
15 |
called programs go into the same cgroup which makes it a bit simpler. |
16 |
also openrc will start your services in the correct cgroup too |
17 |
> this seem to be limited by the manual parsing and CLI actions that |
18 |
> are necessary for large/busy environments. (We shall see). |
19 |
> |
20 |
>> hopefully this will give you a bit more control over all of that though |
21 |
> |
22 |
> Gmane mandates that the previous lines be culled. That said; you have given |
23 |
> me much to think about, test and refine. |
24 |
> |
25 |
> In /sys/fs/cgroup/cpu I have: |
26 |
> |
27 |
> cgroup.clone_children cgroup.procs cpu.shares release_agent |
28 |
> cgroup.event_control cgroup.sane_behavior notify_on_release tasks |
29 |
> |
30 |
> So I'll have to research creating and priotizing dirs like "high_priority" |
31 |
> |
32 |
> |
33 |
> I certainly appreciate your lucid and direct explanations. |
34 |
> Let me play with this a bit and I'll post back when I munge things |
35 |
> up..... Are there any "graphical tools" for adjusting and managing |
36 |
> cgroups? |
37 |
i thought that htop did this but i was wrong.. it only shows which |
38 |
cgroup processes are in. that would be a killer feature though. |
39 |
> Surely when I apply this to the myriad of things running |
40 |
> on my mesos+spark cluster I'm going to need a well thoughout tool |
41 |
> for cgroup management, |
42 |
especially for non-local systems. other distros have apps such as |
43 |
"cgclassify" which provides some shortcut to managing cgroups -- |
44 |
creation / and moving process in and out |
45 |
you can also have a nohup process that does ps -eLf to search for |
46 |
process you want to classify and move them into the appropriate cgroup |
47 |
for default cgroups you can also use inotify |
48 |
a quick search shows http://libcg.sourceforge.net/ which daemonises this |
49 |
process. |
50 |
all this is a bit hack'n'slash though i appreciate, so if anyone else |
51 |
knows of suitable tools i'd also be interested to hear of them |
52 |
> particularly on memory resources organization |
53 |
> and allocations as spark is an "in_memory" environment that seems |
54 |
> sensitive to OOM issues of all sorts. |
55 |
> |
56 |
> thx, |
57 |
> James |
58 |
> |
59 |
> |
60 |
> |
61 |
> |