Re: [gentoo-dev] Proposed future EAPI feature: FILES whitelist

[Kent Fredric]

On Fri, 16 Sep 2016 17:20:28 +0200
Ulrich Mueller <ulm@g.o> wrote:

> > 3. Each entry in FILES dictates that the given file is "Used by" the
> > ebuild.
>
> Do you mean "file" in its Unix meaning here, i.e. including
> directories? Or only regular files?

I'd say regular files for now. Mostly because I think it should be explicit
if you want to include directory children.

/expressions/ could be expanded during MD5 generation against ${REPO}.../files/ 
path, which would probably work in the case of "${PV}/*" for instance.

But I'm undecided if I like that, only suggesting to get people thinking about 
it as a possibility.

>
> > 4. the FILES variable is expanded and interpreted during package
> > sourcing
>
> > 5. "repoman full" and "repoman ci" must ensure any entry listed in FILES
> > exists
>
> > 6. "repoman full" and "repoman ci" must ensure any use of ${FILESDIR}
> > without a declared FILES variable is a failure.
>
> > 7. Under this future EAPI, the location of where ${FILESDIR} expands to
> > changes to a location within
> > ${PORTAGE_TEMPDIR}/portage/${CATEGORY}/${PF}/files
>
> > 8. The contents of ${PORTAGE_TEMPDIR}/portage/${CATEGORY}/${PF}/files
> > is generated from the source-time $FILES entry by mapping entries from
> > ${repostory_location}, either by symlink or by copy (though copy is
> > preferred to avoid potential side effects), mapping their heirachy
> > exactly ( that is, preserving directory structure )
>
> > 9. Files not listed in/indicated by $FILES are thus unavailable to the
> > ebuild at runtime and will not be seen in ${FILESDIR}, even if they are
> > in ${repository_location}
>
> AFAICS that proposal goes into a direction which is somewhat opposite
> to what we have pursued in EAPI 6. There, we have allowed directories
> as arguments to eapply, in order to simplify application of patchsets.
> Now maintainers would have to list all single files contained in the
> directory again.

This wouldn't change the ability for eapply to do that when used with 
patch-tarballs in SRC_URI, and it wouldn't change the ability to call:

   eapply ${FILESDIR}

It would only change where ${FILESDIR} was and what it contained.

But the use of that against directories in $FILEDIR is slightly more
cause for concern at present, though mostly due to abuse where files/*
are all considered. Such abuses /could/ possibly be made more obvious
with this mechanism.

> Also I am not sure if I like the additional burden imposed on ebuild
> maintainers by requiring double bookkeeping. FILESDIR is already a
> well defined location for the support files needed by a package.

Agreed to an extent. Though I was of the opinion that to a greater extent
at least, such concern cases would simply be relocating the book-keeping
logic for defining the file-sets to be at the FILES location, which would
give you a canonical variable to consume elsewhere.

In theory under the expression system, you could have "FILES=*" which
would basically regress the feature to what it currently is, but then having
the feature at all wouldn't be very useful.

It could be an "opt-in" feature instead of a mandatory one mind, where it would
basically degrade to a default mechanic of "Like a PATCHES array, but repoman
can check it"

>
> > :: Benefits ::
>
> > [...]
>
> > 4. Due to referential integrity, it should be trivial to identify which
> > files are needed by a given ebuild, and which files are now redundant,
> > assisting with keeping the tree pruned.
>
> How does a file in FILESDIR get stale? The typical scenario is that a
> patch will no longer be needed after a version bump and pruning of old
> ebuild versions. I fear that with FILES the problem would simply be
> shifted: instead of forgetting to delete the stale file, people would
> forget removing the patch from the FILES list.
>
> Ulrich

I Know those seem like similar problems, but the subtle difference does stick out
for me.

With the existing scenario the basic assumption is that *every* ebuild
needs a given file, and you have to check every ebuild and make sure
you don't break one.

And this is true *even* if you're a maintainer who tries to keep things tight.

A good maintainer can prune the contents of an ebuilds FILES= list *at
the time they perform the bump*, where as by contrast, you *cant* prune
the files themselves at the time of the bump, because they are still
needed.

So in a sense, the use of a FILES= list allows you to keep things
contextualised, you can mark which you still need, and remove that
which you don't.

Which means when a second maintainer comes along and removes an ebuild,
they don't need to have the knowledge the first maintainer internalised
when they made the bump.

Removing the ebuild will immediately alert them of the presense of
unused files, because they will cease to be indicated by any ebuilds.

So you're keeping the knowledge of "I changed the ebuild" attached to
"I changed which files are needed", which will be known at the same
time.

And portage then communicates that knowledge through time to the person
who removes the old ebuilds, so they know which files are no longer
neeed, without having to check every existing ebuild.

Obviously my entire suggestion is not complete and there's probably a
few details that need to be sorted out before it approaches
properly-useful.

And maybe it will serve us better if it starts out as an elective
instead of a mandatory feature.