[gentoo-user] Re: TCP Queuing problem

[Kai Krakow]

Am Wed, 21 Sep 2016 13:47:28 -0700
schrieb Grant <emailgrant@×××××.com>:

> >> I haven't mentioned it yet, but several times I've seen the website
> >> perform fine all day until I browse to it myself and then all of a
> >> sudden it's super slow for me and my third-party monitor.  WTF???
> >
> > I had a similar problems once when routing through a IPsec VPN
> > tunnnel. I needed to reduce MTU in front of the tunnel to make it
> > work correctly. But I think your problem is different.
>
>
> I'm not using IPsec or a VPN.
>
>
> > Does the http server backlog on the other side? Do you have
> > performance graphs for other parts of the system to see them in
> > relation? Maybe some router on the path doesn't work as expected.
>
>
> I've attached a graph of http response time, CPU usage, and TCP
> queueing over the past week.  It seems clear from watching top, iotop,
> and free than my CPU is always the bottleneck on my server.

What kind of application stack is running in the http server? CPU is a
bottleneck you cannot always circumvent by throwing more CPUs at the
problem. Maybe that stack needs tuning...

At the point when requests start queuing up in the http server, the load
on the server will exponentially rise. It's like a traffic jam on a
multi lane high way. If one car brakes, thinks may still work. If a car
in every lane brakes, you suddenly have a huge traffic jam backlogging
a few miles. And it takes time to recover from that. You need to solve
the cause for "braking" in the first place and add some alternative
routes for "cars that never brake" (static files and cacheable
content). Each lane corresponds to one CPU. Adding just more lanes when
you have just 4 CPUs will only make the lanes slower. The key is to
drastically lower the response times which are much too high if I look
at your graphs. What do memory and IO say?

-- 
Regards,
Kai

Replies to list-only preferred.