1 |
Hello, |
2 |
|
3 |
For a while sources.gentoo.org has been puttering along and its health |
4 |
has slowly declined. We migrated it to some newer shiny hardware in an |
5 |
attempt to mitigate the problem but that did not pan out. 90% (or |
6 |
more) of sources.gentoo.org traffic is crawler bots and not actual |
7 |
humans. That being said; if we cannot serve requests to the bots |
8 |
within our timeouts we serve 500's instead which is never really what |
9 |
we want (particularly when we spent 20s of CPU to calculate 80% of the |
10 |
response only to see the client timeout :/.) |
11 |
|
12 |
The majority of the expensive requests are related to package.mask and |
13 |
use.local.desc queries by crawlers. Like crawling the entire 13000 rev |
14 |
history for package.mask (or similar.) |
15 |
|
16 |
While it is likely we will monkey patch viewvc to be less wasteful; in |
17 |
the meantime I have removed use.local.desc from sources.gentoo.org |
18 |
(and also anoncvs, because they share the same repo.) I hope this is a |
19 |
short term (order of weeks) hack. |
20 |
|
21 |
-A |