| 1 |
On September 20, 2016 4:53:41 PM GMT+02:00, Grant <emailgrant@×××××.com> wrote: |
| 2 |
>>>>>> My web server's response time for http requests skyrockets every |
| 3 |
>>>>>> weekday between about 9am and 5pm. I've gone over my munin |
| 4 |
>graphs |
| 5 |
>>>and |
| 6 |
>>>>>> the only one that really correlates well with the slowdown is |
| 7 |
>"TCP |
| 8 |
>>>>>> Queuing". It looks like I normally have about 400 packets per |
| 9 |
>>>second |
| 10 |
>>>>>> graphed as "direct copy from queue" in munin throughout the day, |
| 11 |
>>>but 2 |
| 12 |
>>>>>> to 3.5 times that many are periodically graphed during work |
| 13 |
>hours. |
| 14 |
>>>I |
| 15 |
>>>>>> don't see the same pattern at all from the graph of all traffic |
| 16 |
>on |
| 17 |
>>>my |
| 18 |
>>>>>> network interface which actually peaks over the weekend. TCP |
| 19 |
>>>Queuing |
| 20 |
>>>>>> doesn't rise above 400 packets per second all weekend. This is |
| 21 |
>>>>>> consistent week after week. |
| 22 |
>>>>>> |
| 23 |
>>>>>> My two employees come into work during the hours in question, and |
| 24 |
>>>they |
| 25 |
>>>>>> certainly make frequent requests of the web server while at work, |
| 26 |
>>>but |
| 27 |
>>>>>> if their volume of requests were the cause of the problem then |
| 28 |
>that |
| 29 |
>>>>>> would be reflected in the graph of web server requests but it is |
| 30 |
>>>not. |
| 31 |
>>>>>> I do run a small MTU on the systems at work due to the config of |
| 32 |
>>>the |
| 33 |
>>>>>> modem/router we have there. |
| 34 |
>>>>>> |
| 35 |
>>>>>> Is this a recognizable problem to anyone? |
| 36 |
>>>>> |
| 37 |
>>>>> |
| 38 |
>>>>> I'm in the midst of this. Are there certain attacks I should |
| 39 |
>check |
| 40 |
>>>for? |
| 41 |
>>>> |
| 42 |
>>>> |
| 43 |
>>>> It looks like the TCP Queuing spike itself was due to imapproxy |
| 44 |
>which |
| 45 |
>>>> I've now disabled. I'll post more info as I gather it. |
| 46 |
>>> |
| 47 |
>>> |
| 48 |
>>>imapproxy was clearly affecting the TCP Queuing graph in munin but I |
| 49 |
>>>still ended up with a massive TCP Queuing spike today and |
| 50 |
>>>corresponding http response time issues long after I disabled |
| 51 |
>>>imapproxy. Graph attached. I'm puzzled. |
| 52 |
>>> |
| 53 |
>>>- Grant |
| 54 |
>> |
| 55 |
>> Things to check for: |
| 56 |
>> Torrent or other distributed downloads. |
| 57 |
>> Download program with multiple download threads |
| 58 |
> |
| 59 |
> |
| 60 |
>There sure shouldn't be anything like that running either on the |
| 61 |
>server or in the office. Is there a good way to find out? Maybe |
| 62 |
>something that would clearly indicate it? |
| 63 |
> |
| 64 |
> |
| 65 |
>> Maybe another proxy running? Esp. as you saw this also with |
| 66 |
>imapproxy. |
| 67 |
> |
| 68 |
> |
| 69 |
>nginx acts as a reverse proxy to apache2 but that's a pretty common |
| 70 |
>config. Nothing else that I know of. |
| 71 |
> |
| 72 |
>- Grant |
| 73 |
|
| 74 |
Any way to find out between which hosts/servers those connections are for? |
| 75 |
That might help in locating the cause. |
| 76 |
|
| 77 |
Eg. which of your desktops/laptops inside your network and where they are trying to connect to. |
| 78 |
|
| 79 |
-- |
| 80 |
Joost |
| 81 |
-- |
| 82 |
Sent from my Android device with K-9 Mail. Please excuse my brevity. |
Archives