Gentoo Logo
Gentoo Spaceship

Note: Due to technical difficulties, the Archives are currently not up to date. GMANE provides an alternative service for most mailing lists.
c.f. bug 424647
List Archive: gentoo-dev
Lists: gentoo-dev: < Prev By Thread Next > < Prev By Date Next >
To: gentoo-dev@g.o
From: "Robin H. Johnson" <robbat2@g.o>
Subject: status report, 2009/03/14 06h00 UTC
Date: Fri, 13 Mar 2009 18:56:04 -0700
Yes guys, I know that Bugzilla is down.

Last night, while I was sleeping, we got a slew of IPs hitting the
dependency graph generation. This wouldn't have been a problem normally,
but they seemed to hit graphs that took an inordinate amount of memory
to generate with GraphViz (collectively 8GiB of RAM and 32GiB of swap).
This morning, I got into the box, saw some OOMs of the GraphViz
processes, killed off the remaining ones, banned the IPs, and then had
to rush off to a work meeting.

I came back this evening, to find the box not responding again, and my
last SSH shell was painfully slow then just hung - not died, but hung,
the TCP connection is still alive, but the shell isn't responding
(shortly after I had seen a loadavg exceeding 1k).

Whomever attacked it came back I think. And I can't get in to block them
right now.  I've contacted the sponsor so that they can hard reboot the
box for me, but I don't expect any action from them for the next 5-6
hours at least.

Meantime, I'm enacting a plan B, to at least get us some slow Bugzilla
functionality, via the second bugzilla box that normally runs the
background computations (duplicates etc). I do however fully expect
whomever the attacker is to come right back at it - so I'm turning off
the dependency graphs.

Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@g.o
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85
pgpjUkC90cU5D.pgp (PGP signature)
Replies: status report, 2009/03/19 10h00 UTC
-- Robin H. Johnson
Lists: gentoo-dev: < Prev By Thread Next > < Prev By Date Next >
Previous by thread:
Tcl/Tk multi-slot testing
Next by thread: status report, 2009/03/19 10h00 UTC
Previous by date:
Re: devs on IRC (was :Regen2 ( was QA Overlay Layout support ))
Next by date:
Re: Tcl/Tk multi-slot testing

Updated Jun 17, 2009

Summary: Archive of the gentoo-dev mailing list.

Donate to support our development efforts.

Copyright 2001-2013 Gentoo Foundation, Inc. Questions, Comments? Contact us.