| 1 |
Just saw the post on gentoo-dev. It'll be interesting to see the |
| 2 |
postmortem as not being able to update the infra status page during an |
| 3 |
outage is a bit of a problem ;) |
| 4 |
|
| 5 |
Thanks for the reply |
| 6 |
|
| 7 |
Hayley |
| 8 |
|
| 9 |
On Mon, 2021-06-28 at 23:37 -0500, Dale wrote: |
| 10 |
> Hayley wrote: |
| 11 |
> > Hey all, |
| 12 |
> > Since yesterday I've getting 502 Bad Gateway when trying to access |
| 13 |
> > gitweb.gentoo.org and was wondering if anyone else is having the |
| 14 |
> > same |
| 15 |
> > issue? |
| 16 |
> > The gentoo infra status page says that it should be up so I'm not |
| 17 |
> > sure |
| 18 |
> > what could be causing the problem but it's a 502 so I'm guessing it |
| 19 |
> > must be a server side issue? |
| 20 |
> > |
| 21 |
> > Thanks, |
| 22 |
> > Hayley |
| 23 |
> > |
| 24 |
> > |
| 25 |
> > |
| 26 |
> |
| 27 |
> |
| 28 |
> I just read a message on -project I think that was about some upgrade |
| 29 |
> problems and some parts of infra not working correctly. That could |
| 30 |
> be |
| 31 |
> the problem you were seeing. I think fixing the problem is still in |
| 32 |
> progress so may be a few days before everything is back up to 100%. |
| 33 |
> This is the message: |
| 34 |
> |
| 35 |
> > Alec Warner wrote: |
| 36 |
> > > Hi, |
| 37 |
> > > |
| 38 |
> > > Many of you reported that various infra-owned services were not |
| 39 |
> > > functioning properly these past two days. We had some package |
| 40 |
> > > upgrades roll out that were bad, and a subset of services were |
| 41 |
> > > non-functional while we worked through the incident. |
| 42 |
> > > |
| 43 |
> > > (a) The upgrades did not appear to cause any data loss, but |
| 44 |
> > > service |
| 45 |
> > > availability was reduced (500s / 502s for some HTTP services.) |
| 46 |
> > > (b) Some hosts may have had problems running some commands. E.g. |
| 47 |
> > > I |
| 48 |
> > > know that 'grep' didn't work for a while. If you observed errors |
| 49 |
> > > related to missing 'GLIBC_2.33' symbols then this was likely |
| 50 |
> > > related. |
| 51 |
> > > (c) Many replicated services (including infra-status) rely on |
| 52 |
> > > gitweb |
| 53 |
> > > to take updates from git; and gitweb was not up, so they were |
| 54 |
> > > unable |
| 55 |
> > > to receive updates. |
| 56 |
> > > (d) More impacted notes will follow in a postmortem that will |
| 57 |
> > > come |
| 58 |
> > > later this week for this incident. |
| 59 |
> > > |
| 60 |
> > > Thanks to all of you who reported problems and apologies for the |
| 61 |
> > > service disruptions. |
| 62 |
> > > |
| 63 |
> > > -A |
| 64 |
> |
| 65 |
> |
| 66 |
> Hope that helps. Should keep you from looking for the problem on |
| 67 |
> your |
| 68 |
> end at least. |
| 69 |
> |
| 70 |
> Dale |
| 71 |
> |
| 72 |
> :-) :-) |
| 73 |
> |
Archives