Re: [gentoo-dev] [RFC] Forced/automatic USE flag constraints (codename: ENFORCED_USE)

[Alexis Ballier]

On Wed, 14 Jun 2017 00:13:42 +0200
Michał Górny <mgorny@g.o> wrote:

> On wto, 2017-06-13 at 12:27 +0200, Alexis Ballier wrote:
> > On Mon, 12 Jun 2017 21:17:16 +0200
> > Michał Górny <mgorny@g.o> wrote:
> >
> > > I've actually started typing the initial specification yesterday
> > > [1]. As you can see, banning the extra constraints has made the
> > > algorithms much simpler. In particular:
> > >
> > > 1. You do not have to define 'falsify' for anything other than
> > > pure flags -- which makes it easy to inline it.
> > >
> > > 2. ||, ??, ^^ groups are only flat lists of flags -- which makes
> > > reordering and processing them trivial.
> > >
> > > 3. The algorithm is recursive only on USE-conditional groups. This
> > > makes it trivial to make it iterative. Optimizations become
> > > trivially possible.
> >
> >
> > While you're right in one sense, you're mixing two different things
> > here. What you wrote *is* recursive. It does not recurse just
> > because you're assuming a restricted syntax. You're only saving two
> > things: you don't need to define how to enforce to false (that is 3
> > lines not 3 pages :=) ) and you're avoiding the nested use
> > conditionals that are already ill defined per the current spec
> > (foo? bar is equivalent to && ( foo bar ) when nested) which I
> > believe is already another problem.
> >
> > Then, remember how I wanted to be much more drastic than you in the
> > beginning by killing all ||,&&,^^ etc. and keep only use
> > conditionals in REQUIRED_USE ?  Well, that's where the complexity
> > comes. The whole deal then is to define rewriting rules for the AST
> > so that the algorithm you describe executes the exact same
> > instructions but the new AST only has use conditionals. This is
> > more like writing a compiler for the spec, so this does not belong
> > to the spec and there is no issue here.
>
> I'm looking for a compromise here. Killing those groups completely is
> going to make things harder for users. Keeping them with functionality
> limited to what's used in ~99.9% ebuilds (based on your numbers) is
> IMO a better choice.

I already said I see the limited syntax as a good thing because it
forces devs to write constraints that have a natural interpretation in
how it is solved. However, you can't limit the syntax without a new
EAPI, and more importantly, properly solving does not even require
limiting the syntax.

BTW, I don't know how you get that info from my data because I never
voluntarily checked for a restricted syntax :)

> > [BTW: checking the rewrite rules behave properly is what I meant by
> > rebasing solve() on top of it and being happy with it]
>
> Could you reiterate the current solving rules (trueify/falsify)? Are
> they equal to the ones you listed last, or does the current
> implementation change anything?

Let's recap a bit. nsolve() is poorly named and does not solve
anything. It translates to implications and checks whether the
implications solver will always provide a valid result in one pass.
So, if you only care about solving rules, read your spec man. For the
more general case it should behave like those trueify/falsify with
the change that nested implications are interpreted as && (so no
more !(a -> b) crap to worry about).

If you take solve() as an implementation of your spec, you have:
solve(x) <=> solve(to_impl.convert_to_implications(x)) when solve(x)
is defined; with the added benefit that
'solve(to_impl.convert_to_implications(x))' is defined and should
provide proper results on the whole REQUIRED_USE syntax as currently
defined (granted that nsolve(x) does not report anything wrong).

> > > Nevertheless, feel free to play with the full implementation. If
> > > you're interested, you can play with the complex cases more. In
> > > particular, I'm wondering whether nsolve will actually consider
> > > most of them solvable.
> > >
> > > As for the results, I think it is the point where we start
> > > preparing pull requests with REQUIRED_USE changes to see whether
> > > the developers agree with such changes.
> >
> > If by that you also include code cleanup and writing tests then
> > yes :)
>
> I'm not sure if we're talking about the same thing. I'm talking about
> filing pull requests against ebuilds whose REQUIRED_USE is rejected by
> nsolve. I think it'd serve as a reasonable field test for whether
> developers agree with the imposed restrictions.

I was talking about PRs against portage & repoman.

> > > [1]:https://wiki.gentoo.org/wiki/User:MGorny/GLEP:ReqUse
> >
> > I really don't like the reordering thing. Even the restricted
> > syntax does not fix the issue with '^^ ( a b ) b? ( a )' already
> > mentioned here. It'd be much better and simpler for the spec just to
> > assign a fixed value and use the solving rules with those.
>
> You're not going to convince me by providing examples that are utterly
> broken by design and meaningless ;-).

Well... if it's so obvious that the example is broken by design that
you don't even bother to explain why, I assume you have an algorithm for
that. Where is the code ? What are the numbers ? How many ebuilds might
fail after reordering ? How can this be improved ?

Extra question: Is there *really* a point in pushing user preferences
that way, esp. when developers can write '!b? ( a )' instead of '|| ( a
b )' and just kill any possibility of changing the order ?

As for a real world example, I'll let you find some more interesting
ones, but this one will probably be interesting to you and is a
good start:

app-text/wklej-0.2.1-r1 ^^ ( python_single_target_pypy
python_single_target_pypy3 python_single_target_python2_7
python_single_target_python3_4 python_single_target_python3_5
python_single_target_python3_6 ) python_single_target_pypy?
( python_targets_pypy ) python_single_target_pypy3?
( python_targets_pypy3 ) python_single_target_python2_7?
( python_targets_python2_7 ) python_single_target_python3_4?
( python_targets_python3_4 ) python_single_target_python3_5?
( python_targets_python3_5 ) python_single_target_python3_6?
( python_targets_python3_6 ) vim? ( ^^ ( python_single_target_python2_7
) )


Hint: It loops as written here. Reordering the ^^ in a proper way makes
it solvable. Putting the 'vim? ( ... )' part first makes it solvable in
one pass.

Extra bonus: If your algorithm solves that in reasonable time,
run it again as if PYTHON_COMPAT also contained
'python3_{7,8,9,10,11,12}'

Alexis.