Re: [gentoo-dev] [RFC] Forced/automatic USE flag constraints (codename: ENFORCED_USE)

[Michał Górny]

On śro, 2017-05-31 at 10:38 +0200, Alexis Ballier wrote:
> > > What if I specifically set USE=-bar in make.conf ? Do we really
> > > want PM to override that without telling me ?
> >
> > Yes. Unless you specifically and explicitly disable that (globally or
> > for USE=bar), in which case the PM would just reject to proceed.
>
>
> Then could you please explain how to get the list of useflags the
> solver is allowed to toggle ? Preferably in a PMS-friendly way (aka no
> USE=foo emerge). It's not clear to me what this would be and is
> mandatory for determinism.
>
> Note that most definitions are acceptable, but there must be one.

If we *really* want to set this for the users, it would simply be
a variable defined in make.profile, e.g.:

  REQUIRED_USE_STRICT="foo bar"

which would mean the solver is globally forbidden from touching those
flags, i.e. if the solution would involve touching them, PM must fail
and request user to manually resolve it.

However, as far as I'm concerned we'd be good at keeping this purely as
user configuration, alike FEATURES=i-do-not-want-automatic-solving.

> > > I believe that, from the ebuild POV, the ternary useflag model is
> > > more appropriate: You have a whole bunch of ways to specify
> > > useflags with portage (make.conf, package.use, profiles, command
> > > line, ...). From the ebuild those are collapsed into 'user input'.
> > > You only have IUSE (with its defaults) and that's what the
> > > auto-solver should play with: those are the flags that can be
> > > toggled.
> >
> > I see your point. However, it's merely a preference problem and we
> > really don't want the constraints to become ternary and/or PM try to
> > force the reverse solutions. That's an easy way to lose
> > predictability.
>
> They're not ternary anymore after processing ebuild: IUSE="foo +bar"
> means instantiate foo as -foo if not specified, and bar as +bar.
> The way I see it, ternary model is useful in the sense that the 3rd
> undefined state is what the solver can toggle, the others are fixed
> (either by user input or e.g. use.mask).
>
> Basically, I see automatic solving of REQUIRED_USE as dynamic IUSE
> defaults. But see above, this might not be the best way.

I'm lost on what you're trying to achieve here. Maybe give a full run-
out based on Portage behavior -- i.e. involving all the places USE flags
can be altered in Portage, and how they're going to affect the result.
Don't forget about USE_ORDER.

> > Besides, I should point out that USE_EXPAND in make.profile
> > and make.conf are strictly binary.
>
> Last I checked IUSE="+use_expand_foo use_expand_bar" worked just as
> useflags.

I'm talking about FOO="bar baz" form which implies -* for all remaining
values (and does not accept '-flag', unless I'm mistaken).

>
> > > > Now, this also means that every constraint that can't be solved in
> > > > this easy fashion is invalid. We want to detect that, and warn the
> > > > developer. Some of those could be tricky. Simple example:
> > > >
> > > >   foo? ( baz ) bar? ( !baz )
> > > >
> > > > This one is invalid because USE='foo bar' requires both 'baz' and
> > > > '!baz' as a solution. Remember that we don't want to do any
> > > > changes besides what's explicitly written there, no guessing.
> > >
> > > Besides that, what makes it invalid ?
> >
> > What makes it invalid is that you can't solve it in a predictable way.
>
> You can fail in a predictable way and ebuild writer can adjust it to
> avoid that.

If the point is to process constraints *automatically*, then failing is
not the desired result. Yes, we can consider that a minor issue/warning
level but it is still an issue. I named it 'invalid' because it prevents
the automatic solving from working which is the purpose of this whole
effort.

>  Again, you *need* to process the constraints in order. '!a?
> ( b ) !b? ( a )' is not deterministic when none of a and b are
> enabled otherwise.

You can't rely on any particular order of constraints, especially when
eclass stacking comes into play. You could try defining some constraint-
sorting algorithm but it's going to be complex and it's usefulness will
be limited anyway due to various kinds of grouping.

> You'll get a message like:
> "
>  The constraint bar? ( !baz )' is violated.
>     bar is enabled because $reason (say, make.conf)
>     baz is enabled because of the constraint 'foo? ( baz )'
> 			      foo is enabled because $reason
> "

You = user or ebuild author? Because per my proposal, this construct
should result in QA error/warning, telling the ebuild writer to use
predictable constraints.

Of course, we could still accept the ebuild and just fail hard on user
side (alike REQUIRED_USE). But that's really getting out of scope.

>
> > >  How is it more invalid than '?? ( foo bar )' ?
> >
> > This would go into:
> >
> >   foo? ( !bar )
>
> Just to be clear: Are you suggesting banning '??' from the syntax or
> simply an internal rewrite for the solver ?

'Internal rewrite', i.e. how '??' would have to be interpreted.

>
> > which gives a single predictable solution:
>
> Then ebuild writer should not write 'foo? ( baz ) bar? ( !baz )' but
> rather: 'foo? ( !bar baz ) bar? ( !baz )', which should cover more
> cases. With USE="foo bar", the message would then be:
> "
>   The constraint 'foo? ( !bar )' is violated.
>       foo is enabled because $reason
>       bar is enabled because $reason
> "
> which is similar to the above except there's one less step for
> explaining the reasons. It's not dramatic but it is, indeed, desirable
> to have simple & clear reasons. I'd say that's more to the argument for
> specifying completely how to solve that and leave those small
> improvements to ebuild writers and/or QA.

Is this message meant to be disabled for the purpose of explaining
automatic decisions, or as an error?

Also, your example (unlike the one I gave in the original mail) prefers
foo over bar. That's the only difference.

>
> [...]
> > > > However, the
> > > > following should be valid:
> > > >
> > > >   foo? ( baz ) bar? ( !foo !baz )
> > > >
> > > > Because now we clearly indicate that USE=bar disables USE=foo,
> > > > and therefore makes the first constraint inapplicable. It clearly
> > > > indicates course of action for all combinations:
> > >
> > > Ok, I now think you're aiming for giving full power to the solver,
> > > overriding user inputs if necessary. Before going further, I think
> > > we should first agree on what are the useflags such a solver can
> > > toggle. I'm not sure 'USE=foo emerge blah' should disable foo
> > > instead of failing for example.
> > >
> >
> > As I said, it's a matter of configuration to decide which flags should
> > be touched, and which not. Of course if we find that necessary, we may
> > go into defining a specific set in the profiles or metadata.
> >
> > However, I would rather focus on getting a PoC solver and checker
> > first, and play with existing constraints to see how it all works.
>
> The solver seems on good tracks, at least from the algorithmic POV. The
> checker, however, is not clear at all to me. The main reason is that to
> determine if the solver will be able to solve it, it needs to know what
> the solver can toggle and what not.
>

The point would be: by definition, the solver should be able to touch
*any* USE flag. If it can't, it mostly implies that you can't use
the automatic solver, and so the case is irrelevant for checking.
Attempting to go beyond that is going to give a lot of complexity
and most likely kill the whole idea.

One thing we need to worry about are masks. We need to think about them.

-- 
Best regards,
Michał Górny