[gentoo-dev] Proposed future EAPI feature: FILES whitelist

[Kent Fredric]

Just an idea that seemed obvious enough and obviously missing.

1. Have, in a future EAPI, a variable (I'm going to call it "FILES" as
a stand-in for now ).

2. FILES contains an array (or perhaps whitespace delimited string) of
entries (or perhaps expressions) relative to
${repository_location}/${CATEGORY}/${PN}/files/

3. Each entry in FILES dictates that the given file is "Used by" the
ebuild.

4. the FILES variable is expanded and interpreted during package
sourcing

5. "repoman full" and "repoman ci" must ensure any entry listed in FILES
exists

6. "repoman full" and "repoman ci" must ensure any use of ${FILESDIR}
without a declared FILES variable is a failure.

7. Under this future EAPI, the location of where ${FILESDIR} expands to
changes to a location within
${PORTAGE_TEMPDIR}/portage/${CATEGORY}/${PF}/files

8. The contents of ${PORTAGE_TEMPDIR}/portage/${CATEGORY}/${PF}/files
is generated from the source-time $FILES entry by mapping entries from
${repostory_location}, either by symlink or by copy (though copy is
preferred to avoid potential side effects), mapping their heirachy
exactly ( that is, preserving directory structure )

9. Files not listed in/indicated by $FILES are thus unavailable to the
ebuild at runtime and will not be seen in ${FILESDIR}, even if they are
in ${repository_location}



:: Benefits ::

1. Referential integrity of this field means basic tooling like repoman
can verify in advance the presence of things that are deemed necessary
for the ebuild to function, and can check for their absense.

2. Said referential integrity can propagate failures to mirror
properly and mistakes by ebuild authors (specifically, naughty ones who
side-step repoman) to escallate to being errors long before the files
are needed by executable code.

3. Due to point 2, the risk of silent failures due to lack of explicit
handling for the files being missing is reduced.

4. Due to referential integrity, it should be trivial to identify which
files are needed by a given ebuild, and which files are now redundant,
assisting with keeping the tree pruned.

5. When maintaining a given ebuild, the use of a variable means that
one can consult the list of files a given ebuild uses canonically, and
thus know which files may need oversight when performing changes, as
the canonical form of the variable can theoretically be computed during
metadata/ generation. ( but probably not wise to just flop that in
there, unless we can guarantee that this subtle change to the md5
format won't cause problems elsewhere ), but tooling can still be
written to spit out a list of FILES= entries. ( Currently, you must
grep the ebuild or approximate grep with your eyes, and that is prone
to errors, especially in cases of anything that uses horror shows like
eblits )

6. Due to this introducing a concept of connectedness between .ebuild/
and files/, this opens a door to a potential emerge feature of
--reinstall-if-files-changed , which would allow people to trigger a
rebuild for any ebuild which may have been subtly changed due to
patches to any of its dependency files. ( Though I'd expect to see
little use of this feature if added, except for weird anal people like
me who disable dynamic deps ;) ) 

:: Mixed Change/Benefit Cases ::

1. Some eclasses presently support a PATCHES= variable, which contain
an ordered list of patches that may be applied, stating the order of
application.

Here, in many cases, PATCHES could be dropped from need and a more
universal syntax of `eapply $( echo ${FILESDIR}/*.patch | sort )` as
generic logic in the default src_patch

Due to the ${FILESDIR} being a synthetic location constructed from the
contents of the FILES= variable, you can get away with doing a lot of
tricks that would otherwise be undesirable in the current situation.

For instance: 

     use foo-feature && eapply ${FILESDIR}/foo-feature/*.patch

Because the nature of what "FILESDIR" contained is already strictly
regulated elsewhere in the ebuild and validated long before the patch
phase.


:: Possible Future Abuses ::

FILES=$( cat ${FIlEDIR}/$PV/series )

Though this would stil be validated by repoman to make sure all entries
in "series" exists, and assuming the data was staticised to the md5
metadata cache, would not be so bad. But .... probably should have
people shot for doing this. 


As always, this feature probably exists in some package manager other
than portage for ebuilds in EAPIs other than the one portage support,
but I have not checked those for feature comparison.