1 |
>>>>> My web server's response time for http requests skyrockets every |
2 |
>>>>> weekday between about 9am and 5pm. I've gone over my munin graphs |
3 |
>>and |
4 |
>>>>> the only one that really correlates well with the slowdown is "TCP |
5 |
>>>>> Queuing". It looks like I normally have about 400 packets per |
6 |
>>second |
7 |
>>>>> graphed as "direct copy from queue" in munin throughout the day, |
8 |
>>but 2 |
9 |
>>>>> to 3.5 times that many are periodically graphed during work hours. |
10 |
>>I |
11 |
>>>>> don't see the same pattern at all from the graph of all traffic on |
12 |
>>my |
13 |
>>>>> network interface which actually peaks over the weekend. TCP |
14 |
>>Queuing |
15 |
>>>>> doesn't rise above 400 packets per second all weekend. This is |
16 |
>>>>> consistent week after week. |
17 |
>>>>> |
18 |
>>>>> My two employees come into work during the hours in question, and |
19 |
>>they |
20 |
>>>>> certainly make frequent requests of the web server while at work, |
21 |
>>but |
22 |
>>>>> if their volume of requests were the cause of the problem then that |
23 |
>>>>> would be reflected in the graph of web server requests but it is |
24 |
>>not. |
25 |
>>>>> I do run a small MTU on the systems at work due to the config of |
26 |
>>the |
27 |
>>>>> modem/router we have there. |
28 |
>>>>> |
29 |
>>>>> Is this a recognizable problem to anyone? |
30 |
>>>> |
31 |
>>>> |
32 |
>>>> I'm in the midst of this. Are there certain attacks I should check |
33 |
>>for? |
34 |
>>> |
35 |
>>> |
36 |
>>> It looks like the TCP Queuing spike itself was due to imapproxy which |
37 |
>>> I've now disabled. I'll post more info as I gather it. |
38 |
>> |
39 |
>> |
40 |
>>imapproxy was clearly affecting the TCP Queuing graph in munin but I |
41 |
>>still ended up with a massive TCP Queuing spike today and |
42 |
>>corresponding http response time issues long after I disabled |
43 |
>>imapproxy. Graph attached. I'm puzzled. |
44 |
>> |
45 |
>>- Grant |
46 |
> |
47 |
> Things to check for: |
48 |
> Torrent or other distributed downloads. |
49 |
> Download program with multiple download threads |
50 |
|
51 |
|
52 |
There sure shouldn't be anything like that running either on the |
53 |
server or in the office. Is there a good way to find out? Maybe |
54 |
something that would clearly indicate it? |
55 |
|
56 |
|
57 |
> Maybe another proxy running? Esp. as you saw this also with imapproxy. |
58 |
|
59 |
|
60 |
nginx acts as a reverse proxy to apache2 but that's a pretty common |
61 |
config. Nothing else that I know of. |
62 |
|
63 |
- Grant |