Gentoo Logo
Gentoo Spaceship




Note: Due to technical difficulties, the Archives are currently not up to date. GMANE provides an alternative service for most mailing lists.
c.f. bug 424647
List Archive: gentoo-dev
Navigation:
Lists: gentoo-dev: < Prev By Thread Next > < Prev By Date Next >
Headers:
To: Gentoo Dev <gentoo-dev@g.o>
From: Alec Warner <antarus@g.o>
Subject: sources.gentoo.org instability
Date: Sun, 4 Dec 2011 23:10:09 -0800
Hello,

For a while sources.gentoo.org has been puttering along and its health
has slowly declined. We migrated it to some newer shiny hardware in an
attempt to mitigate the problem but that did not pan out. 90% (or
more) of sources.gentoo.org traffic is crawler bots and not actual
humans. That being said; if we cannot serve requests to the bots
within our timeouts we serve 500's instead which is never really what
we want (particularly when we spent 20s of CPU to calculate 80% of the
response only to see the client timeout :/.)

The majority of the expensive requests are related to package.mask and
use.local.desc queries by crawlers. Like crawling the entire 13000 rev
history for package.mask (or similar.)

While it is likely we will monkey patch viewvc to be less wasteful; in
the meantime I have removed use.local.desc from sources.gentoo.org
(and also anoncvs, because they share the same repo.) I hope this is a
short term (order of weeks) hack.

-A


Replies:
Re: sources.gentoo.org instability
-- Andreas K. Huettel
Navigation:
Lists: gentoo-dev: < Prev By Thread Next > < Prev By Date Next >
Previous by thread:
Automated Package Removal and Addition Tracker, for the week ending 2011-12-04 23h59 UTC
Next by thread:
Re: sources.gentoo.org instability
Previous by date:
Re: We need *you* for a USE="selinux" dependency
Next by date:
Re: We need *you* for a USE="selinux" dependency


Updated Jun 29, 2012

Summary: Archive of the gentoo-dev mailing list.

Donate to support our development efforts.

Copyright 2001-2013 Gentoo Foundation, Inc. Questions, Comments? Contact us.