Re: [gentoo-dev] Proposal to undeprecate EGO_SUM

[William Hubbs]

On Fri, Sep 30, 2022 at 04:53:39PM +0200, Florian Schmaus wrote:
> On 30/09/2022 02.36, William Hubbs wrote:
> > On Wed, Sep 28, 2022 at 06:31:39PM +0200, Ulrich Mueller wrote:
> >>>>>>> On Wed, 28 Sep 2022, Florian Schmaus wrote:
> >>> 2.) the number of EGO_SUM entries exceeds 1000 and a Gentoo developer
> >>> maintains the package
> >>> 3.) the number of EGO_SUM entries exceeds 1500 and a proxied
> >>> maintainer maintains the package
> >>
> >> These numbers seem quite large, compared to the mean number of 3.4
> >> distfiles for packages in the Gentoo repository. (The median and the
> >> 99-percentile are 1 and 22, respectively.)
>
> The numbers may appear large when compared to the whole tree, but I
> think a fair comparison would be within the related programming language
> ecosystem, e.g., Golang or Rust.
>
> For example, analyzing ::gentoo yields the following histogram for
> 2022-01-01:
> https://dev.gentoo.org/~flow/ego_sum_entries_histogram-2020-01-01.png
>
>
> > To stay with your example, restic has a 300k manifest, multiple 30k+
> > ebuilds and897 distfiles.
> >
> > I'm thinking the limit would have to be much lower. Say, around 256
> > entries in EGO_SUM_SRC_URI.
>
> A limit of 256 appears to be to low to be of any use. It is slightly
> above the 50th percentile, half of the packages could not use it.
>
> We have to realize that programming language ecosystems that only build
> static binaries tend to produce software projects that have a large
> number of dependencies. For example, app-misc/broot, a tool written in
> Rust, has currently 310 entries in its Manifest. Why should we threat
> one programming language different from another? Will be see voices that
> ask for banning Rust packages in ::gentoo in the future? With the rising
> popularity of Golang and Rust, we will (hopefully) only ever see an
> increase of such packages in ::gentoo. And most existing packages in
> this category will at best keep their dependency count constant, but are
> also likely to accumulate further dependencies over time.

I tend to agree with you honestly. I worked with Zac to come up with a
different proposal which would allow upstream tooling for all languages
that do this to work, but so far it is meeting resistance [1].
I will go back and add more information to that bug, but it will be later
today before I can do that. I want to develop a poc to answer the
statement that these would be live ebuilds if we allowed that.

> And quite frankly, I don't see a problem with "large" Manifests and/or
> ebuilds. Yes, it means our FTPs are hosting many files, in some cases
> even many small files. And yes, it means that in some cases ebuild
> parsing takes a bit longer. But I spoke with a few developers in the
> past few months and was not presented with any real world issues that
> EGO_SUM caused. If someone wants to fill in here, then now is a good
> time to speak up. But my impression is that the arguments against
> EGO_SUM are mostly of cosmetic nature. Again, please correct me if I am
> wrong.

I can't name any specific examples at the moment, but I have gotten some
complaints about how long it takes to download and build go
packages with hundreds of dependencies.

Other than that, I'm not the one who voiced the problem originally, so
we definitely need others to speak up.

William

[1] https://bugs.gentoo.org/833567