Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[james]

On 08/10/2016 07:45 AM, Michael Mol wrote:
> On Tuesday, August 09, 2016 05:22:22 PM james wrote:
>> On 08/09/2016 01:41 PM, Michael Mol wrote:
>>> On Tuesday, August 09, 2016 01:23:57 PM james wrote:
>
>>> The exception is my storage cluster, which has dirty_bytes much higher, as
>>> it's very solidly battery backed, so I can use its oodles of memory as a
>>> write cache, giving its kernel time to reorder writes and flush data to
>>> disk efficiently, and letting clients very rapidly return from write
>>> requests.
>> Are these TSdB (time series data) by chance?
>
> No; my TS data is stored in a MySQL VM whose storage is host-local.
>
>>
>> OK, so have your systematically experimented with these parameter
>> settings, collected and correlated the data, domain (needs) specific ?
>
> Not with these particular settings; what they *do* is fairly straightforward,
> so establishing configuration constraints is a function of knowing the capacity
> and behavior of the underlying hardware; there's little need to guess.
>
> For hypothetical example, let's say you're using a single spinning rust disk
> with an enabled write cache of 64MiB. (Common enough, although you should
> ensure the write cache is disabled if you find yourself at risk of poweroff. You
> should be able to script that with nut, or even acpid, though.) That means the
> disk could queue up 64MiB of data to be be written, and efficiently reorder
> writes to flush them to disk faster. So, in that circumstance, perhaps you'd
> set dirty_background_bytes to 64MiB, so that the kernel will try to feed it a
> full cache's worth of data at once, giving the drive a chance to optimize its
> write ordering.
>
> For another hypothetical example, let's say you're using a parity RAID array
> with three data disks and two parity disks, with a strip length of 1MiB. Now,
> with parity RAID, if you modify a small bit of data, when that data gets
> committed to disk, the parity bits need to get updated as well. That means
> that small write requires first reading the relevant portions of all three data
> disks, holding them in memory, adjusting the portion you wrote to, calculating
> the parity, and writing the result out to all five disks. But if you make a
> *large* write that replaces all of the data in the stripe (so, a well-placed
> 3MiB write, in this case), you don't have to read the disks to find out what
> data was already there, and can simply write out your data and parity. In this
> case, perhaps you want to set dirty_background_bytes to 3MiB (or some multiple
> thereof), so that the kernel doesn't try flushing data to disk until it has a
> full stripe's worth of material, and can forgo a time-consuming initial read.
>
> For a final hypothetical example, consider SSDs. SSDs share one interesting
> thing in common with parity RAID arrays...they have an optimum write size
> that's a lot larger than 4KiB. When you write a small amount of data to an
> SSD, it has to read an entire block of NAND flash, modify it in its own RAM,
> and write that entire block back out to NAND flash. (All of this happens
> internally to the SSD.) So, for efficiency, you want to give the SSD an entire
> block's worth of data to write at a time, if you can. So you might set
> dirty_background_bytes to the size of the SSD's block, because the fewer the
> write cycles, the longer it will last. (Different model SSDs will have different
> block sizes, ranging anywhere from 512KiB to 8MiB, currently.)


Ok, after reading some of the docs and postings, several time, I see how 
to focus in on the exact hardware on a specific system. The nice thing 
about clusters is they are largely identical systems, or groups of 
identical systems, in quantity so that helps with scaling issues.... 
testing specific hardware, individually, should lead to near-optimal 
default settings so they can bee deployed as cluster nodes, later.


>> As unikernels collide with my work on building up  minimized and
>> optimized linux clusters, my pathway forward is to use several small
>> clusters, where the codes/frameworks can be changed, even the
>> tweaked-tuned kernels and DFS and note the performance differences for
>> very specific domain solutions. My examples are quite similar to that
>> aforementioned  flight sim above, but the ordinary and uncommon
>> workloads of regular admin (dev/ops) work is only a different domain.
>>
>> Ideas on automating the exploration of these settings
>> (scripts/traces/keystores) are keenly of interest to me, just so you know.
>
> I think I missed some context, despite rereading what was already discussed.

Yea, I was thinking out loud here. just ignore this...

>>>> I use OpenRC, just so you know. I also have a motherboard with IOMMU
>>>> that is currently has questionable settings in the kernel config file. I
>>>> cannot find consensus if/how IOMMU that affects IO with the Sata HD
>>>> devices versus mm mapped peripherals.... in the context of 4.x kernel
>>>> options. I'm trying very hard here to avoid a deep dive on these issues,
>>>> so trendy strategies are most welcome, as workstation and cluster node
>>>> optimizations are all I'm really working on atm.
>>>
>>> Honestly, I'd suggest you deep dive. An image once, with clarity, will
>>> last
>>> you a lot longer than ongoing fuzzy and trendy images from people whose
>>> hardware and workflow is likely to be different from yours.
>>>
>>> The settings I provided should be absolutely fine for most use cases. Only
>>> exception would be mobile devices with spinning rust, but those are
>>> getting
>>> rarer and rarer...
>>
>> I did a quick test with games-arcade/xgalaga. It's an old, quirky game
>> with sporadic lag variations. On a workstation with 32G ram and (8) 4GHz
>> 64bit cores, very lightly loaded, there is no reason for in game lag.
>> Your previous settings made it much better and quicker the vast majority
>> of the time; but not optimal (always responsive). Experiences tell me if
>> I can tweak a system so that that game stays responsive whilst the
>> application(s) mix is concurrently running then the  quick
>> test+parameter settings is reasonably well behaved. So thats becomes a
>> baseline for further automated tests and fine tuning for a system under
>> study.
>
> What kind of storage are you running on? What filesystem? If you're still
> hitting swap, are you using a swap file or a swap partition?

The system I mostly referenced, rarely hits swap in days of uptime. It's 
the keyboard latency, while playing the game, that I try to tune away, 
while other codes are running. I try very hard to keep codes from 
swapping out, cause ultimately I'm most interested in clusters that keep 
everything running (in memory). AkA ultimate utilization of Apache-Spark 
and other "in-memory" techniques.


Combined codes running simultaneously never hits the HD (no swappiness) 
but still there is keyboard lag. Not that it is actually affecting the 
running codes to any appreciable degree, but it is a test I run so that 
the cluster nodes will benefit from still being (low latency) quickly 
attentive to interactions with the cluster master processes, regardless 
of workloads on the nodes. Sure its  not totally accurate, but so far 
this semantical approach, is pretty darn close. It's not part of this 
conversation (on VM etc) but ultimately getting this right solves one of 
the biggest problems for building any cluster; that is workload 
invocation, shedding and management to optimize resource utilization, 
regardless of the orchestration(s) used to manage the nodes. Swapping to 
disc is verbotim, in my (ultimate) goals and target scenarios.

No worries, you have given me enough info and ideas to move forward with 
testing and tuning. I'm going to evolve these  into more precisely 
controlled and monitored experiments, noting exact hardware differences; 
that should complete the tuning of the Memory Management tasks, within 
acceptable confine  . Then automate it for later checking on cluster 
test runs with various hardware setups. Eventually these test will be 
extended to a variety of  memory and storage hardware, once the 
techniques are automated. No worries, I now have enough ideas and 
details (thanks to you) to move forward.


>> Perhaps Zabbix +TSdB can get me further down the pathway.  Time
>> sequenced and analyzed data is over kill for this (xgalaga) test, but
>> those coalesced test-vectors  will be most useful for me as I seek a
>> gentoo centric pathway for low latency clusters (on bare metal).
>
> If you're looking to avoid Zabbix interfering with your performance, you'll
> want the Zabbix server and web interface on a machine separate from the
> machines you're trying to optimize.

agreed.

Thanks Mike,
James