Re: [gentoo-amd64] Re: Hanging after a few days - 2.6.15-r7

[Brett Johnson]

I'd like to take a minute to thank Duncan, as I usually learn
something new with his very informative posts! Thank you for taking the
time to write such detailed posts, I for one look forward to reading
them.

On Mon, Mar 13, 2006 at 04:25:05PM -0700, Duncan wrote:
> Generally, the first step in tracing such bugs is to see if it still
> occurs with the mainline/vanilla kernels.  If not, it's a patch in
> whatever portage kernel you are using, so a Gentoo bug should be filed.
> If it occurs with mainline, try at least to narrow it down to the rc where
> the problem started, and file a kernel.org bug.  Preferably, narrow it
> down further to the daily git snapshot that  did it.  There are
> instructions for doing that  and even going further, narrowing it down to
> the file and even procedure or line changed between the two git snapshots,
> if you have the time to do so, in the doc/BUG-HUNTING text-file in your
> kernel sources dir.  Narrowing it down beyond the file level takes a lot
> of time and patience, but narrowing it down to the file isn't too bad, and
> narrowing it down to the GIT snapshot isn't hard at all, since tarballs of
> those are made available at kernel.org just as if it were a normal kernel.
>
That is great information, and I will pursue  this if I do indeed have
some memory anomaly.

> You didn't mention this so I'm not sure if you know to make the
> distinction or not -- what sort of memory usage was it?  Application usage
> (indicates a leak) or simply cache or buffer usage?  Linux will normally
> fill most of memory with cache and buffers, as that's more efficient than
> having it remain unused.
>
I did think about documenting my method of determining memory usage, and I
did mention one of them, conky. One if the stats conky displays is RAM
usage.  I also use the free command to view memory

>
> if you aren't already aware of the distinction, learn the
> difference between the different types of "used" memory, cache and buffers
> vs. application memory, in particular.  (FWIW because it's hard to find a
> definition for buffers, the difference between cache and buffers is that
> buffers are basically dedicated cache -- usable by one application not the
> entire system, while cache is systemwide. That's not absolutely accurate,
> but it's a practical/working definition. The two are often considered
> together and should be, as their effect on memory is similar and
> cumulative.)
>
I have been under the assumption that the "-/+ buffers/cache" line in the free
command shows the actual memory "used" by the kernel and applications, and the
"free" memory available for cache/buffers. This is how I have always based my
assessment of memory usage.

Here is a sample from this morning with just mutt, irssi, gaim and aterm
running with fluxbox as the wm:

             total       used       free     buffers	cached
Mem:       1028780     996560      32220     234996	202852
-/+ buffers/cache:     558712     470068
Swap:       979956        224     979732

I have thought that me "Mem: used" should always be as high as possible,
meaning I am using as much ram in the system for applications as well as
cache and buffers. I assumed the" -/+ buffers/cache used" is how much
memory is consumed by the kernel and running applications. The "-/+ used 
number" is the one that I have seen grow overnight unusually high. This
is also the number that conky seems to report on. Prior to "the change",
and after a reboot, running the normal applications listed above, I
average about 120000 on the "-/+ buffers/cache used". Maybe I am
interpreting this data incorrectly, and I in fact do not have a problem.

As a side bar, I saw mention in this list before, I think by Duncan, a
"recommended" book on the Linux kernel. Unfortunately, it seems I deleted
that thread, so if someone knows a good book on the kernel and memory
management, I would appreciate any recommendations.

> Kernel 2.6 has a swappiness tweaking control ( /proc/sys/vm/swappiness ),
> that determines the balance between keeping stuff cached and swapping out
> more applications, once all memory is used.  This  is set by default to
> 60.  If you want more applications kept in memory at the expense of
> caching, set it lower, 0 means always prefer keeping applications in
> memory even if they aren't used, which was the 2.4 series default.
> Conversely, higher than 60 will favor cache more than applications, so
> apps will tend to be swapped out quicker while cache is maintained.  I
> have a four-disk RAID setup here, with swap at the same priority over all
> of them, so my swap is pretty fast and I keep swappiness set to 100,
> favoring cache over application memory some of which won't be used very
> often anyway.  (I'm running RAID-6 for much of my system, for 2X
> redundancy.  Thus, on a four-disk array, it's two-way striped, while swap
> is four-way striped, so reading apps back in from swap will be faster than
> having to read data back in off the RAID-6 if it has been flushed out of
> cache, and swappiness=100 is more efficient.  Normal non-RAID single disk
> usage would mean swap and file rereading off of disk would be roughly the
> same speed, so the best performance would be closer to that 60 default
> swappiness.)
>
Again, I learned something new today (and it's not even 8am here!), and
I always kind of wondered why Duncan had some much swap and how he utilized it.
This makes sense, and I will try tweaking this setting, as normally I don't
 use any swap.
-- 
gentoo-amd64@g.o mailing list