[gentoo-project] towards a more distributed model

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 11/15/2014 09:29 PM, Rich Freeman wrote:
> On Sat, Nov 15, 2014 at 10:00 AM, hasufell <hasufell@g.o>
> wrote:
>>
>> Turning it off will decrease organizational problems, make the
>> distribution more focused and eventually more high quality,
>> because people would be able to actually work together on the
>> _core_ of gentoo (toolchain, base-system, eclasses, PM...).
>>
>
> Having fewer people maintaining packages will not make more people
> work on toolchain, base-system, eclasses, portage, etc.
>
> If somebody is actively contributing to something in critical need
> such as one of those I'd certainly encourage the recruiters to
> prioritize their application.  However, turning away help
> elsewhere won't help us out.
>
> I'd suggest moving forward with building the new before getting rid
> of the old.
>

I'm not saying you should agree with me, but I get the feeling you
didn't get the idea. I'm also answering to the other mail from
Matthias Dahl in here.

To reiterate, more gentoo _core_ developers also means:
* more different opinions
* less focus
* more bikeshed
* more conflict
* less work done?


People cannot easily group around an idea in gentoo and just start
hacking on it, because:
* although we have overlays, gentoo isn't designed to be very modular
* it's tools are not fit for more abstract approaches
* some non-core ideas/projects are already implemented in the main
tree (more or less) which are sometimes disconnected from the
community and may block progress and explicitly conflict (also on the
ebuild level, very tedious to handle) with other approaches
* it's a monolithic all-in-one distro, so anyone who likes the core of
the distro, but wants to build extensions around it has to effectively
do a full fork. Full forks are less likely, so we are not giving too
much room for experiments, enhancements and totally new ideas. Sure,
there are a few of that kind, but it took considerable effort for
these. Instead, it should be made easy... and we should try to learn
from such experiments. I don't see that happening much.

As was already pointed out: quite a lot of gentoo projects are already
moving very slowly in that very direction. They are mainly working on
themed github hosted overlays with frequent user contributions.

I may know more about ebuild writing than some community users, but it
often happens that they know way more about actual packages than me.
They rarely find the way to contribute through all these tedious
barriers we put up for them (bugzilla, cvs, getting your social status
before you can actually get stuff done)... which does NOT improve
ebuild quality.

We are imposing a workflow on people, although that workflow might
just be wrong in general or in certain cases.
That's why some gentoo projects have already a dual-workflow. Why not
make it more open?

You may say "it is open", really? Our main channel is GLEPs (uagh),
users rarely participate on gentoo dev ML.

You may say "we already allow conflicting ideas". Well yes, that's
part of the problem! We allow conflicting projects/ideas in ONE
repository That part of the GLEP never got into my head, and I know
why. It's not the way to get diversity and interesting experiments...
by giving only one box to 200 people and tell them you may practically
do what you want. It just leads to inconsistencies, fights and a big
mess, because we didn't care about proper abstraction, but about
implementing our diverse, but specific ideas right now.
No, you'd rather give everyone the same box and tell him he may do
with that as he wants. He may find out there are other people with the
exact same ideas, so there goes.


So the idea is the following:
* stop recruiting
* move clearly themed overlays/projects out of the tree which are
already practically working outside of the tree (e.g. science,
haskell, ...)
* fix the tools (never ending git story, probably a lot of work needed
on the overlay support front etc)
* focus on the core of gentoo as in: provide abstraction, tools and
the basic structure for people to do cool things. Right now we are
just focussing on keeping the ebuild machinery going, while the rest
pretty much goes downhill. Fast.
* be more open, work more with the community, not just through bugzilla
* review overlays, contribute to overlays
* have a list of high-quality overlays, maybe with a few notes about
them (is it themed? does it conflict with stuff?)
* I can't hold it but say: make ebuilds suck less, so people enjoy
contributing


But... what about quality?
Well, IMO we have lots of very poor-quality ebuilds in our tree. Not
just in overlays. That's because it's still hard to contribute, so a
small number of people have to do a LOT of work (as in: actually
writing the ebuilds) and we have no review workflow.
People who are so experienced with ebuilds like gentoo old timers with
5+ years of experience probably shouldn't write ebuilds AT ALL. Their
time is better spent just reviewing ebuilds, either by request or just
to check the quality of an overlay.

Doing things a bit more distributed will rather improve ebuild
quality, as long as we have some sort of review culture. So that's
something for the community to chew as well. But that already works in
some cases. We see it happening.
I know AUR and I think it's terrible, because I've been an arch linux
user for 2 years. So that's something we have to avoid and learn from
if we try to go into this direction.


But... what about security?
That's a more delicate matter, yeah. I can understand the concerns,
but lets start with this question first:
Why do you trust ME? You don't know anything about me. All you know is
that I probably didn't mess up too hard for the last 2 years and that
I regularly sign my Manifests/commits.
Well, a lot of overlay maintainers do as well (I once asked the
tox-overlay maintainers to gpg sign their commits and so they did).
So IMO it just boils down to community reputation and verifiable
authorship. Why care then if the guy committing is an actual "gentoo
dev" if his reputation is good?

Besides that... webapps in gentoo are in terrible shape (not blaming
anyone... tried to write such ebuilds myself, it can be awfully
complex). I'm running a gentoo server myself and I ended up installing
a lot of web apps directly from git repositories instead of the
in-tree packages. It would probably be interesting if a group from the
community will step up and clear these things out with the help of a
few gentoo devs. Or maybe a company will be interested in maintaining
a high-quality overlay of server tools and webapps? Why not? There are
already companies recruiting for gentoo experienced people and using
gentoo for servers. So why not do that?

What's more important on the security side is a different thing imo:
* our GLSA channels. They will not be obsoleted by any of these ideas.
* tools like glsa-check
* high responsiveness (in my experience a lot of overlay maintainers
are faster with fixing vulnerable packages than we are, but yeah...
this will also require more communication)

Also... no one is saying we have to remove apache from the list of
core packages.


So... why do it this way?
Because a lot of successful projects exactly operate like that.
Including VERY big projects: have a very limited number of core
developers who make the core decisions and have a strong focus. Then
have the community input and openness to get influenced both by code
and ideas while still allowing people to easily deviate from some
ideas, because we keep to the core.

I don't think we are "community-driven", just because we keep adding
community members to our "ranks". Community-driven means for me that
there is a very strong connection and communication between the core
developers and the community.
Most of the time... this doesn't ever happen and decisions are made on
our dev ML which is mostly bikeshed between gentoo devs!

In addition, if we focus on the core... we don't even have to make
that many decisions any more.

This really requires a shift of thinking and I honestly don't expect
that to happen.
But it's something that is happening around as and seems to work out
quite well (latest example is NixOS, without wanting to talk about
their technical decisions).
Some distros have programs to analyse these projects and try to
integrate some of those ideas into their own.

But I'm not really interested in fighting the decisions other people
made about gentoo ~10 years ago and write 55 GLEPs to change
something. I'd rather spend my time improving overlays or find people
(in gentoo) who think similar.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0

iQJ8BAEBCgBmBQJUaUAGXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w
ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQzMDlCNDQ4NjEyNDI4NjA5REVEMDI3MzIy
MjBDRDFDNUJERUVEMDIwAAoJECIM0cW97tAgq6UP/ioha93yY/F/ryh8XH7mkymB
Wjwn4+L1mAV/i9YD8J70IpcbPKRmj3QQItcFVCPb8mLGFCdOr4G2LoVhRObH10IE
jqJC4QvyNL9chvsRSHvPWyFUkVGSqcgT/SlhPmtsuJt2EvSglEeC0hAXvL+YDOSL
zotuT3gETYCTnpc5xqOjkEWSSW7dHT6aA2PeXqmcNukPo4UpfJEKz06KEu7lUpER
nlWiPGL/XyxlVRB2LLNHyCHlX3dYQvZeVCXs1MFbKB1XMwNpQthWAzaJ+4ADKGVN
vd2IeByviivdkNQCc5nby6jHFZlMUiWdJJLJfdb0G39Hz2wHCXD8Crx4leqDvV3e
5OethLFq/QMV3D9gPnwMr7JWVisEAIJzCXeCyNbFnhQuA3bp9ZP6rFd8ndwt3Xzm
fQMoTjdZE279qbMbv4mrXer7/4+rIs0wPK88Bqd31HediQgsvKgHNnVx4z9gUIrn
O/rdGsV8vg9EA6bLTWBNkvXCJ38eh+8n583CXzy1FEC5esUa2y7o4iZ8MrPzUw2A
+Cc7ovSnk+bvj/XXG2ptbCC4tltuJcwFg4r+QjGEXVEpAqfh/4zgYQUIKta1ff2a
f72vvZPkR4MNC3tznWSS9SWVQL27RAzDqb4qWnRpj/sggwLcCDlYNH4WLCETkao3
G+lBZsQmmtyvyYpkrxV6
=VFTg
-----END PGP SIGNATURE-----