Gentoo Archives: gentoo-user

From: Mick <michaelkintzios@×××××.com>
To: gentoo-user@l.g.o
Subject: Re: [gentoo-user] Re: Dealing with scrapers - Help!
Date: Sat, 09 Aug 2008 16:26:23
Message-Id: 200808091726.17258.michaelkintzios@gmail.com
In Reply to: [gentoo-user] Re: Dealing with scrapers - Help! by Grant
1 On Saturday 09 August 2008, Grant wrote:
2 > > My apache web server has been very slow lately and webalizer charts
3 > > show page accesses at 5x normal with other stats normal. I'm thinking
4 > > scrapers? How do you guys deal with this? Do you identify the IP
5 > > (how?) and ban it (how?)?
6 > >
7 > > - Grant
8 >
9 > I used netstat to identify the IP and I see that I can use it with
10 > "deny from" in httpd.conf. It seems to be over now, but this type of
11 > thing happens periodically. How can I be alerted to this type of
12 > situation when it starts so I can block the IP right away?
13
14 You will need to configure quotas probably using something like:
15
16 http://www.howtoforge.com/mod_cband_apache2_bandwidth_quota_throttling
17
18 Not sure if it is possible to differentiate between rogue and legit clients,
19 other than by checking your logs to see what was blocked.
20
21 HTH.
22 --
23 Regards,
24 Mick

Attachments

File name MIME type
signature.asc application/pgp-signature

Replies

Subject Author
Re: [gentoo-user] Re: Dealing with scrapers - Help! Grant <emailgrant@×××××.com>