| 1 |
Rich Freeman <rich0 <at> gentoo.org> writes: |
| 2 |
|
| 3 |
|
| 4 |
|
| 5 |
|
| 6 |
> A big problem with Linux along these fronts is that we don't really |
| 7 |
> have good mechanisms for prioritizing memory use. You can set hard |
| 8 |
> limits of course, which aren't flexible, but otherwise software is |
| 9 |
> trusted to just guess how much RAM it should use. |
| 10 |
|
| 11 |
Exactamundo! |
| 12 |
Besides fine grained controls I want it in a fat_boy controllable gui! |
| 13 |
Clustering is where it's at. NOW much of the fuss I read |
| 14 |
in the clustering groups, particularly Spark and other |
| 15 |
"in_memory" tools, is all about monitoring and managing |
| 16 |
all types of memory and related issues. [1] |
| 17 |
|
| 18 |
|
| 19 |
> It would be nice if processes could allocate cache RAM, which could be |
| 20 |
> preferentially freed if the kernel deems necessary. If some pages are |
| 21 |
> easier to regenerate than to swap, this could also be flagged (I have |
| 22 |
> a 50Mbps connection - I'd rather see my browser re-fetch pages than go |
| 23 |
> to disk when the disk is already busy). There are probably a lot of |
| 24 |
> other ways that memory use could be optimized with hinting. |
| 25 |
|
| 26 |
I think you need to look into apache spark. It is exploding. Technology |
| 27 |
to run certain codes 100% in memory looks to be a revolution, driven |
| 28 |
by the mesos/spark clusters. [2] The weapons on top of mesos/spark |
| 29 |
are Python, Java and Scala (in portage). |
| 30 |
|
| 31 |
|
| 32 |
hth, |
| 33 |
James |
| 34 |
|
| 35 |
[1] https://issues.apache.org/jira/browse/SPARK-3535 |
| 36 |
|
| 37 |
[2] https://amplab.cs.berkeley.edu/ |
| 38 |
|
| 39 |
http://radar.oreilly.com/2014/06/a-growing-number-of-applications-are-being-built-with-spark.html |
Archives