Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Michael Mol]

On Tuesday, August 09, 2016 01:23:57 PM james wrote:
> On 08/09/2016 09:17 AM, Michael Mol wrote:
> > On Tuesday, August 09, 2016 09:13:31 AM james wrote:
> >> On 08/09/2016 07:42 AM, Michael Mol wrote:
> >> > On Monday, August 08, 2016 10:45:09 PM Alan McKinnon wrote:
> >> >> On 08/08/2016 19:20, Michael Mol wrote:
> >> >>> On Monday, August 08, 2016 06:52:15 PM Alan McKinnon wrote:
> >> >>>> On 08/08/2016 17:02, Michael Mol wrote:

> > I use Zabbix extensively at work, and have the Zabbix agent on my
> > workstation reporting back various supported metrics. There's a great
> > deal you can use (and--my favorite--abuse) Zabbix for, especially once
> > you understand how it thinks.
>
> Congradualtions! Of the net-analyzer crowd, you've manage to find one I
> have not spent time with........

Oh, man, are you in for a treat. I recently had a conversation with a guy I 
happened to sit next to while traveling about how, were I in his position, I'd 
improve his cash crop and hydroponics operations (he periodically tests soil 
and sunlight properties) continually using a combination of cheap, custom 
probes and SBCs, feeding the data into Zabbix for monitoring and trend 
analysis / prediction. Zabbix will do time-series graphing and analysis of 
arbitrary input data; it may have been designed for watching interface 
counters, but there's no reason it need be limited to that...

>
> >> Any specific kernel tweaks?
> >
> > Most of my tweaks for KDE revolved around tuning mysqld itself. But for
> > sysctls improving workstation responsiveness as it relates to memory
> > interactions with I/O, these are my go-tos:
> >
> >
> >
> > vm.dirty_background_bytes = 1048576
> > vm.dirty_bytes = 10485760
> > vm.swappiness = 0
>
> Mine are::
> cat dirty_bytes
> 0
> cat dirty_background_bytes
> 0

So, that means you have vm.dirty_bytes_ratio and vm.dirty_background_ratio 
set, instead. I forget what those default to, but I think 
dirty_bacgkround_ratio defaults to something like 10, which means *10%* of 
your memory may get used for buffering disk I/O before it starts writing data 
to disk. dirty_bytes_ratio will necessarily be higher, which means that if 
you're performing seriously write-intensive activities on a system with 32GiB 
of RAM, you may find yourself with a system that will halt until it finishes 
flushing 3+GiB of data to disk.

> cat swappiness
> 60

Yeah, you want that set to lower than that.

>
> > vm.dirty_background_bytes ensures that any data (i.e. from mmap or
> > fwrite, not from swapping) waiting to be written to disk *starts*
> > getting written to disk once you've got at least the configured amount
> > (1MB) of data waiting. (If you've got a disk controller with
> > battery-backed or flash-backed write cache, you might consider
> > increasing this to some significant fraction of your write cache. I.e.
> > if you've got a 1GB FBWC with 768MB of that dedicated to write cache,
> > you might set this to 512MB or so. Depending on your workload. I/O
> > tuning is for those of us who enjoy the dark arts.)
> >
> >
> >
> > vm.dirty_bytes says that once you've got the configured amount (10MB) of
> > data waiting to be disk, then no more asynchronous I/O is permitted
> > until you have no more data waiting; all outstanding writes must be
> > finished first. (My rule of thumb is to have this between 2-10 times the
> > value of vm.dirty_background_bytes. Though I'm really trying to avoid it
> > being high enough that it could take more than 50ms to transfer to disk;
> > that way, any stalls that do happen are almost imperceptible.)
> >
> >
> >
> > You want vm.dirty_background_bytes to be high enough that your hardware
> > doesn't spend its time powered on if it doesn't have to be, and so that
> > your hardware can transfer data in large, efficient, streamable chunks.
> >
> >
> >
> > You want vm.dirty_bytes enough higher than your first number so that
> > your hardware has enough time to spin up and transfer data before you
> > put the hammer down and say, "all right, nobody else gets to queue
> > writes until all the waiting data has reached disk."
> >
> >
> >
> > You want vm.dirty_bytes *low* enough that when you *do* have to put that
> > hammer down, it doesn't interfere with your perceptions of a responsive
> > system. (And in a server context, you want it low enough that things
> > can't time out--or be pushed into timing out--waiting for it. Call your
> > user attention a matter of timing out expecting things to respond to
> > you, and the same principle applies...)
> >
> >
> >
> > Now, vm.swappiness? That's a weighting factor for how quickly the kernel
> > should try moving memory to swap to be able to speedily respond to new
> > allocations. Me, I prefer the kernel to not preemptively move
> > lesser-used data to swap, because that's going to be a few hundred
> > megabytes worth of data all associated with one application, and it'll
> > be a real drag when I switch back to the application I haven't used for
> > half an hour. So I set vm.swappiness to 0, to tell the kernel to only
> > move data to swap if it has no other alternative while trying to satisfy
> > a new memory allocation request.
>
> OK, OK, OK. I need to read a bit about these. Any references or docs or
> is the result of parsing out what is the least painful for a
> workstation? I do not run any heavy databases on my workstation; they
> are only there to hack on them. I test db centric stuff on domain
> servers, sometimes with limited resources. I run lxde and I'm moving to
> lxqt for workstations and humanoid (terminal) IO.

https://www.kernel.org/doc/Documentation/sysctl/vm.txt
https://lonesysadmin.net/2013/12/22/better-linux-disk-caching-performance-vm-dirty_ratio/

>
>
> Do you set these differently for servers?

On my servers, I keep these values similar, because I'd rather have a little 
bit lower throughput than risk a catastrophic cascade failure stemming from an 
I/O stall.

>
> Nodes in a cluster?

Same story.

The exception is my storage cluster, which has dirty_bytes much higher, as 
it's very solidly battery backed, so I can use its oodles of memory as a write 
cache, giving its kernel time to reorder writes and flush data to disk 
efficiently, and letting clients very rapidly return from write requests.

>
> I use OpenRC, just so you know. I also have a motherboard with IOMMU
> that is currently has questionable settings in the kernel config file. I
> cannot find consensus if/how IOMMU that affects IO with the Sata HD
> devices versus mm mapped peripherals.... in the context of 4.x kernel
> options. I'm trying very hard here to avoid a deep dive on these issues,
> so trendy strategies are most welcome, as workstation and cluster node
> optimizations are all I'm really working on atm.

Honestly, I'd suggest you deep dive. An image once, with clarity, will last 
you a lot longer than ongoing fuzzy and trendy images from people whose 
hardware and workflow is likely to be different from yours.

The settings I provided should be absolutely fine for most use cases. Only 
exception would be mobile devices with spinning rust, but those are getting 
rarer and rarer...

-- 
:wq