Re: [gentoo-guis] Why I don't like catapult

[Ciaran McCreesh]

On Tue, 01 Apr 2008 16:11:10 +0200
René 'Necoro' Neumann <lists@××××××.eu> wrote:
> >> Ok. Perhaps we can postpone this until it gets real. (And then
> >> perhaps use scpvr's ;) ... like: cpan:bla-foo/SomePerlPkg-0.13)
> >
> > Paludis does it already.
>
> So we can set it as the package format? (Though needing further
> definitions.)

Mm? With Paludis you get things like CRAN as 'just another repository'.

> > How does Paludis know which configuration set it should be using to
> > respond to queries?

When you invoke a Paludis client you can use --environment to specify
a configuration set.

> Ok - so we need sessions ...
>
> I see two possibilites to implement it:

The third is to have different DBus services per configuration.

> That's true. But I want it for three reasons:
> - - no security issues
> - - safe way is more important than nice way: In case the information
> catapult provided (e.g. dep tree) is not correct (for whatever
> reason), I don't want it to crash the users system. Thus, it is
> better to let the package manager to have the final say. - This
> reason gets obsolete if the providers are maintained by the package
> manager teams themselves, but this is not the case atm.
> - - at least in portage I can't see a way to say "hey - portage:
> install the following packages please", as the logic behind this is
> done in emerge itself.
>
> Is there any other reason to vote against this besides "isn't the
> nice way"?

Yeah. You can't use it to write a decent client. More on that below.

> > Doesn't make much difference either way... Why is the revision the
> > only version part you treat specially?
>
> We could also return "" instead "-r0" if no revision is given. Does
> not make a difference.

Why split revision at all?

> Now I see, why you want the user to know of the EAPI... Do you think
> it would be enough to allow to query the EAPI value of a package? Then
> (taking this example) the client can decide whether to ask for
> "DEPEND" or "DEPENDENCIES" depending on the EAPI.
> If for some reason, he asks for "DEPEND" and the package uses EAPI2 an
> EAPIError or something similar is being thrown.

The issue with doing things that way is that every single client has to
be updated for every new EAPI.

It's probably useful to look at how Paludis and Portage handle EAPIs.
Portage basically just has some hardcoded if-else logic ("if the EAPI
is 1 then ...") in relevant areas (and doesn't generally enforce EAPI
things). This has the advantage that it's quick to code if you have a
small number of relatively similar EAPIs. It has the disadvantage that
when we introduce a new EAPI someone has to go through and check all
those code paths, and add a bunch more. It doesn't scale well to large
changes.

Paludis does things differently. EAPI-dependent code never directly
queries EAPI. Instead, it queries a central class for capabilities
(things like "does this EAPI support slot deps?" or "does this EAPI
use an ECONF_SOURCE-aware src_compile?"). And rather than hardcoding
configuration keys ("get me the DEPEND string" and so on), we have an
abstracted high-level interface ("get me the key that holds
dependencies that have been parsed into an abstract higher level
format") that's sufficiently powerful to deal with all EAPIs, along with
non-ebuild packages. This means that clients written using the Paludis
EAPI (and, indeed, the core Paludis library) have absolutely no mention
or awareness of EAPI anywhere, and yet always behave correctly for all
EAPIs.

Forcing every client to be EAPI aware and updated for every new EAPI is
majorly sucky. On the other hand, the Paludis approach quite possibly
only works because we have strong static checking and rich classes...

> > The package manager handles || on a case by case basis. Anything
> > containing || cannot be flattened to a simple list.
>
> When installing the the package the package manager is taking one of
> the choices of "||", based on certain information. A similar approach
> should be used here too.

That certain information is the key issue. That certain information, in
the dependency resolution case, is a rather complex context that can
include things like speculative branch taking on earlier || cases,
speculative unmasking and knowledge of packages that aren't installed
yet but that will be by the time the ID providing the || is reached for
install.

For example... The Paludis handling code for || ( ) deps when resolving
a normal dependency goes very roughly like this:

- Is the || ( ) block empty after removing use? blocks? If so, pretend
it's empty.

- Try to rewrite the || ( ) block to a single multi-operator dep spec,
and use that instead of the || ( ) block if we can.

- Are any of the || ( ) children already installed? (Note that this
handles complex blocks and the like too...) If so, take that child.

- If we have something like || ( a >=b-2 ) and b-1 is installed, take
the b-2 branch if we can do so.

- Go for the first viable option.

- Fall back and go for the first option to get an exception.

You can't emulate this logic by flattening.

> >> Just return some kind of an URL. The end tool has to deal with it.
> >
> > Then the name needs changing, and it needs to be documented that the
> > return value is entirely meaningless.
>
> Name change ok - perhaps s/path/url/ ?
>
> "entirely meaningless" - I would use "might be meaningless" instead.

So what's the URL for what Paludis calls VirtualsRepository or
InstalledVirtualsRepository?

> > But they can't write to cache on disk. Remember that Portage needs
> > you to be in the 'portage' group to do anything, and anyone in the
> > 'portage' group can easily obtain root on the box.
>
> Nope - you can query even if you are not in the portage group.

Really not a good idea...

> Despite the current open issues (btw: I tried to collect them on a
> wiki page: http://catapult.origo.ethz.ch/wiki/current_issues ), do
> you think that the whole system can be brought to something useful in
> the end?

I think you have one big issue: you don't have a clear purpose for the
API.

On the one hand, you're saying that it should be a nice simple API for
clients.

On the other hand, you're saying that you want to use it for writing a
nice fancy GUI.

Of the Paludis clients we've experimented with, the one that makes by
far the most complex use of the API is the GUI client. This isn't
merely because we can -- the only time a GUI becomes useful is when it
offers functionality that can't easily be provided quickly by a non-GUI
client. A GUI should be able to do things like display a resolution of
'update world with deps', and have little clicky things on each item
saying things like "exclude this from the update" (greyed out if the
item can't be excluded), "show me a tree of why this package is
included in the list" and "select a different item from the || ( )
branch that pulled this dep in". You can't do this without an extremely
rich API (and you probably can't do it without lambdas hooked into the
resolver...). You definitely can't do this by having a crude "give me a
command I can exec()" way of installing things.

-- 
Ciaran McCreesh