Re: [gentoo-dev] [pre-GLEP] Optional runtime dependencies via runtime-switchable USE flags

[Brian Harring]

On Sun, Jun 17, 2012 at 10:31:59PM +0200, Micha?? G??rny wrote:
> Hello,
>
> A simple solution to a program long-unsolved. In GLEP form.
>
> Both attached and published as a gist:
>
> https://gist.github.com/2945569
>
> (please note that github doesn't render GLEP headers correctly)
>
> --
> Best regards,
> Micha?? G??rny

> GLEP: XXX
> Title: Optional runtime dependencies via runtime-switchable USE flags
> Version: $Revision:$
> Last-Modified: $Date:$
> Author: Micha?? G??rny <mgorny@g.o>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 17 Jun 2012
> Post-History:
>
>
> Abstract
> ========
>
> This GLEP addresses the issue of referencing optional runtime
> dependencies in Gentoo packages and ebuilds. It does introduce
> a concept of runtime-switchable USE flags to achieve that goal.
>
>
> Motivation
> ==========
>
> Optional runtime dependencies are often found in packages installing
> various scripts (shell, python, perl). These are not strictly required
> for the particular package to work but installing them enables
> additional functionality.
>
> Unlike in compiled programs, enabling or disabling those features
> (dependencies) does not affect the files installed by the package.
> They can be installed and uninstalled independently of the package,
> resulting in changes of functionality without a need to rebuild
> the package.
>
> Currently such dependencies are usually expressed only through
> ``pkg_postinst()`` messages. This forces user to manually install
> the necessary dependencies, and uninstall them when they are no longer
> necessary.
>
> Another solution is using regular USE flags. Those flags do not strictly
> follow the principles of USE flags because they do not affect files
> installed by the package and are not entirely effective to the package
> (a disabled feature will still be available if necessary dependency is
> installed). Additionally, it requires unnecessary rebuilds
> of the package in order to change the dependencies.
>
>
> Specification
> =============
>
> The ebuilds aiming to provide features enabled through optional runtime
> dependencies should:
>
> 1. create regular USE flags for all those features, following
>    appropriate specifications for Gentoo ebuilds, and including
>    the flags in the ``IUSE`` variable;
> 2. introduce additional ``IUSE_RUNTIME`` variable listing names of USE
>    flags related to optional runtime dependencies (without prefixes
>    related to IUSE defaults).
>
> Additionally, the ebuilds must obey the following rules:
>
> 1. all flags listed in ``IUSE_RUNTIME`` have to be listed in ``IUSE``,
> 2. flags listed in ``IUSE_RUNTIME`` can be referred in ``RDEPEND``,
>    ``PDEPEND`` and ``REQUIRED_USE`` variables,
> 3. flags listed in ``IUSE_RUNTIME`` must not be referred in phase
>    functions, ``DEPEND`` or ``SRC_URI``,
> 4. flags listed in ``IUSE_RUNTIME`` may be referred through USE
>    dependencies by other packages' ``DEPEND``, ``RDEPEND``
>    and ``PDEPEND`` variables.

Unless I'm on crack, you're stating that essentially an optional use 
flag (one you label 'runtime'), is able to be used transitively during 
DEPEND.  That's not particularly "here's some suggested pkgs to 
install"- that's "rebuild the fucker for this changed DEPEND", which 
is the existing situation.


> The package manager should treat flags listed in ``IUSE_RUNTIME``
> as regular USE flags, except for the following:
>
> 1. the state of the flags must be re-evaluated each time the package
>    dependency graph is considered,
> 2. enabling or disabling any of the flags must not involve rebuilding
>    the package,
> 3. the flags may be listed in the visual output in a distinct way
>    to inform the user that they affect runtime dependencies only.
>
>
> Rationale
> =========
>
> The proposed solution tries to solve the issue of handling runtime
> dependencies while reusing the existing infrastructure. Most
> importantly, users will be able to reuse the existing tools
> and configuration files to enable and disable optional runtime
> and build-time dependencies alike.
>
> The remaining reused features include:
>
> - dependency syntax,

If you invent new syntax, I'm going to be unhappy.  KISS as it were.

> - ability to use ``REQUIRED_USE``, USE dependencies,
> - ability to describe flags in `metadata.xml`,
> - global flag names (and descriptions).
>
> Alternative proposed solution involved creating additional ``SDEPEND``
> variable. That proposition had the following disadvantages:
>
> - being package-oriented rather than feature-oriented,

No; use flags are our configuration space, and they turn on/off 
sections of the given pkgs graph.  Your proposal relies on the same 
concept; bluntly, what you're proposing is just as 'package oriented'.

Effectively, you can't dismiss SDEPEND/ODEPEND via changing the rules 
between your proposal and ODEPEND's proposal.  Nice try though. :)


> - lack of ability to express multiple packages required by a single
>   feature,

Eh?  SDEPEND="my_feature? ( pkg1 pkg 2 )"


> - lack of ability to express cross-feature dependencies,

Clarify.  This is a wee bit too vague for responding to ;)



> - lack of ability to describe features provided by enabled packages,

metadata.xml's local use description already covers that; in your 
proposal above you lock in on use flags for the controlling of it 
(which, presumably you'll abuse to get descriptions of) yet claim it's 
not possible for ODEPEND (better name than SDEPEND :P).


> - necessity of implementing a new user interface parts to control
>   the dependencies,

Um... This is bullshit.  Your proposal above is going to require a lot 
more implementation, documentation of how this all is supposed to 
work, and user ededucation for the weird scenarios where things don't 
behave as expected, or they don't understand why changing use flag X, 
results in the PM pulling in new packages (acting akin to -N), while 
changing flag Y doesn't, and only does so/rebuilds via -N.

That's not one of those things we can easily summarize in a UI; not 
unless we want to extraordinarily verbose.


> - lack of backwards compatibility.

This is a bit of a stretch; to be clear, you're proposing that the 
depgraph changes in subtle/nasty ways on the fly under your scheme, 
and that a PM supporting IUSE_RUNTIME vs one that doesn't, won't find 
new and subtle ways to blow up the packages involved (specifically to 
expose differing behaviour).


As ciaran said, I'd like to see an actual implementation if you're 
dead set on this route; ODEPEND/SDEPEND, not so much; it's 
straightforward.  Your scheme basically requires the manager to 
continually recalculate large swaths of resolution subgraphs (every 
time), doesn't account for the fact that most PMs actually write a 
finalized form of the deps to the DB (for efficiency reasons, and 
since the USE state is locked), requires that optimization to be 
*reversed* basically everywhere.

Mind you, I'm generally a bit of a harsh reviewer; I appreciate the 
notion/effort/emphasis, I just think bluntly that this is overly 
complex, and you're dismissing the saner/simpler solution w/out 
sufficiently strong reasons (especially in light of the complexity of 
the alternative, and consequences on PM behaviour/performance/user 
confusion).  I could be wrong- feel free to blast back.

~harring