Re: [gentoo-dev] Preserving mtimes for EAPI3

[Ulrich Mueller]

>>>>> On Mon, 30 Mar 2009, Marijn Schouten (hkBst) wrote:

> On behalf of the Lisp project (which includes the Emacs subproject)
> I'd like to propose that preservation of mtimes be included as a
> requirement of EAPI3.

> [...]

> Background: Dynamic languages such as Common Lisp and Elisp, but
> also python (and ruby?) compile source files to some form which
> loads and executes faster; in Lisp-speak these are called fasl's
> (for FASt Load), for python these are the pyc files. Need for
> recompilation is determined by comparing the mtimes of the original
> source and the fasl. Both source and fasl are usually installed.
> If the mtimes are modified such that the fasl is not newer than
> the original source anymore then implementations will attempt to
> recompile these sources and will try to write the output alongside
> the sources. This will cause a sandbox violation if it happens in
> an ebuild or fail due to permissions when done as a user.

I'll try to summarise the current state of discussion in
<https://bugs.gentoo.org/264130>. The goal is to satisfy two
(apparently contradictory) requirements:

  a) Some packages need a preserved ordering of file modification
     times, therwise things will break at run time (see above).

  b) Files in the installed system should not have mtimes that are
     older than the build time of the package. 

Currently, Portage and Pkgcore preserve file modification times when
merging, which doesn't always fulfil b). Paludis updates mtimes, which
breaks a).

Now, is it possible to satisfy both? I think that the following
procedure would accomplish it:

  1. Record two timestamps:
     before calling pkg_setup, timestamp1;
     after src_install has completed, timestamp2.

  2. After src_install and before merging (the exact time would be
     implementation dependent), scan ${D} for files having
     mtime < timestamp1 or mtime > timestamp2.
     Update their mtimes to timestamp1 or timestamp2, respectively.

  3. Otherwise (i.e. for files with timestamp1 <= mtime <= timestamp2),
     preserve modification times when merging ${D} to ${ROOT}.

This way, any files generated during the build process (*.pyc, *.fasl,
*.elc) would end up with timestamps newer than their corresponding
source files (*.py, *.lisp, *.el).

Problems remain for packages with pre-compiled files, or for packages
requiring exact mtimes (like ghdl). But I think this affects only very
few packages, and it could be fixed on the ebuild level. For example,
a Lisp package with both *.lisp and precompiled *.fasl could touch its
fasl files at the end of src_install.


I am aware of the fact that we are late for EAPI 3 (partly because I
didn't expect that the change would require an EAPI bump). Question to
the council: is it still possible to include this? Considering that
there is a lot of breakage, as well as strange workarounds related to
the current inconsistent behaviour of package managers.

Ulrich