Re: [gentoo-dev] [gentoo-project] Re: towards a more distributed model

[hasufell]

On 11/20/2014 04:03 AM, Rich Freeman wrote:
> On Wed, Nov 19, 2014 at 7:36 PM, hasufell <hasufell@g.o> wrote:
>>
>> But keep in mind that the core is supposed to shrink with this idea of a
>> distributed model! So it would be less work to actually roll/tag
>> releases than it would be right now (or even do that stuff in branches).
>
> This doesn't really make the issue go away.  Are all dependencies
> going to have to stay in the core?

no, why?

> That is a lot of packages, and
> we'll need lots of controls so that updates to those dependencies
> don't break all of these distributed repos whose existence we may not
> even be aware of.
>

You are exaggerating, imo. We already have these breakages in the tree,
because a lot of teams do not communicate, including security bug fixes
on libraries that break 20+ packages of a different subsystem (including
stable packages, multiple times).
Also, no one tests all reverse deps when bumping a package in ~arch.

Whether these packages live in one repository or several almost doesn't
make any difference, as long as there is communication.

It took me 5 minutes to communicate that libsdl2 is not going to be
slotted in the tree to all major overlays that had libsdl:2 packaged.
They fixed it within 30minutes. That's ~50 times faster than what I
expect in the tree.

It's not really an issue to have communication channels and a community
infrastructure for those things.

Sure, these things sound scary if you imagine overlays with 10+ overlay
dependencies.
But first of all, you don't really depend on an overlay just because you
need a few packages from it. If it matters where that package actually
comes from, then there is either something wrong with your own package
(e.g. because it requires a hacked version of a lib) or something is
wrong with the list of high-quality overlays, because some of them ship
broken/hacked libraries.

It's up to the PM to give suggestions on possible dependencies/overlays
that contain the missing package. (that's what paludis already does)

> Or, maybe we allow the dependencies to be in the overlays.  Now we
> have the issue that there might be 5 different editions of that
> floating around, and different packages depend on different ones, but
> they're not intended to co-exist.  Oh, and you still have the problem
> that if the dependency changes it could break anything that uses it.
>

It's debatable whether that is an actual technical problem to be solved
or just a reason to mark that overlay low quality.

>
>>
>> Exherbo is already running a more modular approach, I'd be interested
>> what they have to say about this or which problems they were facing.
>> I'll probably also dig a bit deeper into NixOS and see what tools they
>> use for creating NixOS based distros.
>>
>
> I'm not sure how Exherbo is doing things, I'm certainly interested in
> their comments.
> From what I've read about Nix, it basically installs
> every package-version into its own directory tree, so if zlib is in 75
> different repositories, and has 2 SONAMEs in use, then you'll have up
> to 150 copies of it installed on your system each in its own directory
> tree, assuming you have 150 packages each using a different fork of
> it.  I'll optimistically assume that only half of them contain
> security issues.  :)
>
> That is my issue with some of these more devops-oriented approaches
> that try to bypass the centralized distro.  They make things really
> easy for developers, but they seem almost impossible to control from a
> patching/security/integration standpoint.  People have brought up
> things like Android and iOS and their framework model as a solution,
> but all the examples of this I've seen are all focused on the
> presentation layer alone.  It is great for a mobile app that doesn't
> do anything but open sockets back to a server and display widgets.  If
> you're actually building the part of an application that does the real
> logic/work, then suddenly you find yourself caring about a lot more
> than list controls.  Good luck finding a framework that includes
> everything in /usr/portage/*-libs
>

You lost me here. It's pretty clear that there will be some more amount
of "what should my overlay actually be about?" involved.

I'm sure the maintainer of an overlay will know that 5+ overlays use
packages from it and it will very likely become a joint effort, because
everyone is interested that the dependencies are ok.

I see a lot of things to figure out (especially PM-side, tools-side,
maybe even PMS, maybe even repository layout, but also documentation and
if it makes sense culture-wise), but I don't see a fundamental
unsolvable problem here.