[gentoo-dev] Re: Re: example conversion of gentoo-x86 current deps to unified dependencies

[Steven J. Long]

On Sun, Sep 30, 2012 at 03:15:18PM -0700, Brian Harring wrote:
> On Wed, Sep 19, 2012 at 09:16:02AM -0400, Ian Stakenvicius wrote:
> > On 19/09/12 09:09 AM, Michael Orlitzky wrote:
> > > The problem appears as we introduce more DEPEND variables (which is
> > > what prompted the proposal, IIRC). If we have ADEPEND, BDEPEND,
> > > CDEPEND, and DDEPEND, and there's only some (i.e. not total)
> > > sharing going on then the COMMON_DEPEND pattern starts to fall
> > > apart. You potentially need,
> > >
> > > AB_DEPEND AC_DEPEND AD_DEPEND BC_DEPEND BD_DEPEND CD_DEPEND
> > > ABC_DEPEND ABD_DEPEND ACD_DEPEND BCD_DEPEND ABCD_DEPEND
> > > (COMMON_DEPEND)
> > >
> > > This obviously gets worse as more DEPEND vars are introduced.
> > >
> >
> > Well not really, no -- the additional *DEPENDs that are being proposed
> > (or at least mentioned) for new EAPI will either remove atoms from
> > COMMON_DEPEND/DEPEND/RDEPEND or will be used so tersely that a
> > COMMON_DEPEND or other intermediate variable won't really be necessary
> > for them.
>
> It depends on the dep type actually, and how we're viewing those deps-
> if they must be complete or not.
>
> Consider test depends for example.

I'd like to see some concrete discussion of link-depends/lib-depends, since
they represent a much more fundamental type of dependency, and would imo
remove atoms from a lot of other places.

Discussing test dependencies first, seems like putting the cart before the
horse.

> The point I'm trying to make here is that each dep phase
> should be authorative; in doing so, you start getting
> a lot of potential subsets (DEPEND is a subset of TDEPEND, TDEPEND
> isn't completely, but mostly a subset of RDEPEND as RDEPEND is a
> likely a superset of DEPEND; PDEPEND is a superset of RDEPEND).
>
And all of them are a superset of LDEPEND (which is why I say it's more
fundamental, and class it as an omission from the original conversion
from ports to portage. The fact that link-dependencies are required to be
installed before build can begin, does not make them BUILD_DEPENDS, of
which DEPEND is the equivalent in Gentoo.)

In terms of what has been mentioned before, the set of dependencies
required at a particular point, LDEPEND are required at all of them.

As for the authoritative set at each point, you've already shown that the
package mangler has to build that currently, so there's no implementation
cost in continuing to do so.

> So... you could do COMMON_DEPEND, COMMON_TDEPEND, COMMON_RDEPEND in
> the ebuild.  Or you could just use a syntax form that allows you to
> directly inline that up front, rather than having to muck around w/
> intermediate vars.
>
Or you could address the gaping hole in the spec, and allow people to simply
specify the most fundamental dependency, without having to hack it in.

Then TDEPEND would just be what was required in addition during TEST phases,
a large amount of those common dependency variables would not be needed, and
similarly for C/HOST_DEPENDS (aka the new DEPEND under the awfully-specified
HDEPEND EAPI) since by definition a library dependency lives in the same
ROOT as the installed package.

>
> > Besides, this isn't actually a -problem- as there's nothing which
> > really requires one to use such helpers; ebuild writers just, well,
> > can.  :)
>
> Getting to this point; yes, people could hack around it manually.
> Pretty sure that hasn't been in doubt.
>
> The question is what do we *gain* from making them do so.  No one here
> has stated what we gain from COMMON_DEPENDS in comparison to
> DEPENDENCIES (where they can just inline it).
>
LDEPEND enables much better tracking of linkage, which is pretty fundamental
to building and maintaining a distribution. It's also something that
programmers specify upfront in configure and makefiles. As shown above,
it eliminates a lot of common dependencies, and at least some HDEPEND (if
that variable name is used accurately.)

Additionally, separating out the variable (which yes, can also be done with
a label, but it does need to be done) means linkage can be tracked
automatically based on the sort of data that every package supplies, every
other distribution provides, and every developer has to think about when
writing an ebuild; instead of on yet another syntax obfus^W complication that
was designed for java and python plugins.

Doing it in a separate variable as opposed to labelling it, means that
variable can be treated differently, should that be desired in a new EAPI.
Such a change in how LDEPEND were specified, would not impact on any of the
rest of the single variable format (which is more complex, and more
interdependent, at least in its scripted form) and would not need to be
verified against all the other types of specification in the rest of that
"integrated" variable. So a separate variable is more flexible.

It's also a *lot* more transparent.

For instance, if we'd had that before, do you agree we'd have been scanning
binaries to check for linkage outside that set and system? (After all, we
scan for a lot of other stuff already; scanelf is a gentoo success story.)

How would usage of revdep-rebuild been affected? What about implementation
of preserved-libs?
 
> And as I'm trying to lay out, where dependencies *should* be going
> towards, this issue will get worse if we're doing it out of band from
> DEPENDENCIES.
>
You've given a convincing argument for the package mangler using a unified
variable internally, and if the PM authors all agree, using that in the
cache format. (ie the extra parsing complexity and maintenance is worth it
for the difference in IO across the cache.)

But you've also shown exactly how all that information can be inferred from
the existing ebuild format (and once inferred would actually be simpler than
the whole variable specified in the ebuild.)  As such, it appears more like
something that should be in the back-end, without requiring such a shift in
how people write ebuilds.

Your additional arguments as to why this gets more complex later down the
line are predicated on ignoring LDEPEND.

If that requires a bit more runtime at cache-generation, we're already paying
that price. Saving a small amount of time there, where most users never see
it, doesn't seem worth the mental effort, and maintenance cost, required
to change the format. "Just go ahead and use it in the cache," seems like
the best solution.

Regards,
steveL.