Gentoo Archives: gentoo-project

From: Alec Warner <antarus@g.o>
To: Gentoo Dev <gentoo-dev@l.g.o>, gentoo-project <gentoo-project@l.g.o>
Subject: [gentoo-project] [FYI] Gentoo Service instability June 28 - 29
Date: Tue, 29 Jun 2021 03:12:47
1 Hi,
3 Many of you reported that various infra-owned services were not functioning
4 properly these past two days. We had some package upgrades roll out that
5 were bad, and a subset of services were non-functional while we worked
6 through the incident.
8 (a) The upgrades did not appear to cause any data loss, but service
9 availability was reduced (500s / 502s for some HTTP services.)
10 (b) Some hosts may have had problems running some commands. E.g. I know
11 that 'grep' didn't work for a while. If you observed errors related to
12 missing 'GLIBC_2.33' symbols then this was likely related.
13 (c) Many replicated services (including infra-status) rely on gitweb to
14 take updates from git; and gitweb was not up, so they were unable to
15 receive updates.
16 (d) More impacted notes will follow in a postmortem that will come later
17 this week for this incident.
19 Thanks to all of you who reported problems and apologies for the service
20 disruptions.
22 -A