[gentoo-dev] Breakage and frustration

[Patrick Lauer]

Broken breakage


tl;dr: Stuff is broken, and no one seems to care



In August the git "migration" happened, moving our main repository from
old stupid cvs to modern shiny git.

Well, migration is not the word I'd use, because this was an untested
forced migration that is now, months later, still suffering from
regressions and failures. But, hey, no more cvs! That's good because,
reasons.

So at first there was [1] the lack of proper Manifests. Which broke
things for all rsync users for  a few hours.

Once that was 'fixed' there was the fun of [2], which made emerge --sync
very expensive because it refetched lots of files. Every time.

The 'fix' to the fix of the fix for that is still in progress ...


And a few little things like [3] happened. Oopsie.
Also there seems to be confusion how git works, leading to hilarity like
[4].

Now [5] was reported. Who needs ChangeLogs, this is git! Except for all
users, who don't get ChangeLogs. Well, let's add them and [6] not test
what happens. Guess what? Stuff breaks more.
And they are added backwards so that emerge --changelog fails in a
different way. No, I didn't want to read all changlog except the part I
cared about.

And fixing that introduces [7] some more regressions that broke updating
@system for about 3.5 days.

The fix to that fix (notice a pattern here?) broke rsync for *all* users
[8]. Almost as if no one ever tests things in a test environment ... but
hey, we're agile, let's fix stuff in production!

And the manifest issues are still [9] making life exciting.

So to summarize, in about 5 months there was user-visible breakage:

- ~1 day downtime for git migration (no updates)
- 8h no Manifests (no updates possible)
- a few days of emerge --sync being stupidly slow
- a few days of emerge-webrsync not updating
- about 3 months of emerge --changelog being broken, just to be broken
in a different way
- 3.5 days of emerge @system being broken
- about a day of emerge --sync needing manual interaction to be able to
update again
- a few days of grub being uninstallable (iow, making installing
impossible for many users)

So all in all emerge --sync && emerge -uND @system being down for >10%
of the time.

Now, I don't know if you use Gentoo, but I do, and I use it at work, so
having this level of randomization happen is not really useful.

Tell me then, please - what can I/we do so that this kind of breakage
stops, and we can actually aim at having a most excellent distro? In the
long run I am considering just creating my own clone of all
infrastructure bits so I can fix things, but it's an option that is
needlessly braindead, wasting effort, and not really useful to users
that are not me.


[1] https://bugs.gentoo.org/show_bug.cgi?id=557184
[2] https://bugs.gentoo.org/show_bug.cgi?id=557192
[3] https://bugs.gentoo.org/show_bug.cgi?id=557344
[4] https://bugs.gentoo.org/show_bug.cgi?id=557400
[5] https://bugs.gentoo.org/show_bug.cgi?id=557826
[6] https://bugs.gentoo.org/show_bug.cgi?id=565574
[7] https://bugs.gentoo.org/show_bug.cgi?id=565694
[8] https://bugs.gentoo.org/show_bug.cgi?id=567074
[9] https://bugs.gentoo.org/show_bug.cgi?id=567830