Re: [gentoo-guis] Why I don't like catapult

[Ciaran McCreesh]

On Mon, 31 Mar 2008 00:37:31 +0200
René 'Necoro' Neumann <lists@××××××.eu> wrote:
> Hmmm - ok then virtuals really need to be thought about.

Or just forcibly ignored...

> >>> * use of regexes throughout. Every language defines its own,
> >>> different regex dialect. There's no portability between languages.
> >> True - I thought of using some kind of globbing - thus only
> >> allowing "*" and "?" (and perhaps "^" and "$")
> >
> > Why support it at all? That's not something that should be in at the
> > package manager level.
>
> That's true ... but catapult is a bit higher than package manager
> level. And at least the "find_" functions should allow them, to make
> searching and alike simpler.

Sounds like you need a layered architecture. A low level catapult to
package manager layer, and a high level package manager independent
extras layer.

> Hmm ... no. I see you are coming from the package-manager theory side.
> And there EAPI is important. But I can't see any point here where EAPI
> plays a role (besides package-IDs (which can be resolved using
> catapult API versions)), that can't be covered _inside_ the provider.

As soon as you query any kind of information or perform any kind of
information on an ID your code must be EAPI aware.

> I don't doubt that e.g. the catapult-paludis provider has to deal with
> EAPI in some way. But the _user_ (or better: application dev) using
> catapult API does not need to be aware of it. Catapult is of higher
> level than package management itself.

But application developers do have to be aware of EAPI! Otherwise they
aren't allowed to query anything about the IDs they receive.

> Ok. Perhaps we can postpone this until it gets real. (And then perhaps
> use scpvr's ;) ... like: cpan:bla-foo/SomePerlPkg-0.13)

Paludis does it already.

> > Paludis supports multiple configurations. You can, for example,
> > have /etc/paludis, /etc/paludis-mychroot and
> > ~/.paludis-anotherchroot, and Paludis can operate with all of thse.
> > Paludis can also operate on systems where there's no configuration
> > -- for example, you shouldn't need a full config setup just to be
> > able to run QA checks on one particular repository.
> >
> > Portage also sort of has some of this in a very crude way, through
> > PORTAGE_CONFIGROOT.
>
> As catapult only queries, this can be resolved _inside_ the paludis
> provider.
> This of course gets hairy if it comes to libcatapult to provide
> config-file manipulation...

How does Paludis know which configuration set it should be using to
respond to queries?

> A "set" is a "set of packages" for me. You mention a "set of specs" -
> don't get what kind of "specs" you are referring to here.

A set isn't a set of packages. It's a set of dependency specifications
(foo/bar or >=foo/bar-2 or whatever).

> >>> - find_packages. As find_best_match for search key. What
> >>> disambiguation is performed upon the atom? What happens if you
> >>> pass it something like 'git', which is ambiguous?
> >> Return everything that fits. It should be on the users side to
> >> translate it into something useful.
> >
> > That's messy.
>
> Why? - It's a search function. And I guess in most cases the "return
> everything that matches" is in case intended.

Have a look at what existing tools do when given an ambiguous package
name part. They generally error, not do everything. Coding various
things gets tricky when you have to manually check that you've only
been given a single package name.

> >>> - get_config_path. Unportable. What if the user is using some
> >>> other config format?
> >> But the configs (read: package.mask, package.use etc.) have to be
> >> at a specified unique location, don't they?
> >
> > For Paludis? No. NoConfigEnvironment, for example, uses no
> > configuration directory at all.
> >
>
> This really is an issue... And what is happening if the user wants to
> set an option (e.g. a useflag) in the "NoConfigEnvironment"? - Is a
> config directory created and the environment switched?

No. Users can't set options in NoConfigEnvironment. There is literally
no configuration beyond the constructing parameters.

> Nevertheless, I want to provide applications with the ability to run
> a package manager with different option switches which follow some
> global semantics. Or do you think, something like this is just not
> possible and we have to look for another way of doing so?

Doing this via an exec() is the wrong way to do it. If you want it to
be sanely usable you need to do it via a proper API.

> Think of the following use case: An application provides an interface
> that allows users to select a package for installation. Now this
> application uses catapult and wants to query for the command which has
> to be run to get this package installed. Using get_merge_command, the
> app could do:
>
> spawn(catapult.system.get_merge_command(), package_list)
>
> Catapult isn't the system merging here - it just provides the
> information to actually _do_ the merge.

Which is the wrong way of doing it. You might as well ask for a command
that outputs DESCRIPTION for a given ID rather than providing an API
call to do it directly.

> >>> - split_cpv. Why the weird revision thing?
> >> What's so weired about it?
> >
> > Why is the revision considered special?
>
> I'm waiting what's the result of the "-r0"-thread on gentoo-dev before
> giving an anwer ;)

Doesn't make much difference either way... Why is the revision the only
version part you treat specially?

> We have the following cases:
> 1.) "DEPEND" is a fixed catapult term/constant. Then the provider has
> to map to "DEPENDENCIES" if it encounters the term and EAPI2 is used.

Not doable. DEPENDENCIES can't map to DEPEND. DEPEND isn't powerful
enough syntactically.

> 2.) "DEPEND" is directly passed to the package manager (for example
> using "get_package_settings"). I think this isn't preferred behavior
> and only a given set of constants for the "settings" and alike should
> be used. -> see 1.)

Then you'd get a spurious blank result.

> >>> And what's the return value? What about when || is in operation?
> >> WWPMD (what would the package manager do): Simplified: Pass the
> >> dep-string to the package manager and see what he does with it. The
> >> result is returned.
> >
> > The package manager has to have special handling for || throughout.
> > The behaviour of || blocks is rather tricky.
>
> I guess it is a major problem if the package manager isn't able to
> handle "||", right? - So I can't see your point here.

The package manager handles || on a case by case basis. Anything
containing || cannot be flattened to a simple list.

> >>> - get_iuse_flags. What does this do for IUSE defaults? For
> >>> installed packages, all flags are forced.
> >> defaults are returned verbatim. I used to define "forced flags" as:
> >> "The flags the user can't change." (like userland_GNU or
> >> use.mask'ed flags)
> >
> > The user can't change any flags for installed packages.
>
> True. But he can re-install it using the changed flags. (Though he
> isn't using the flags of the installed packages here, but the ones of
> the uninstalled and overwriting the installed one.)
> In other words: For this use case, the useflags of installed packages
> are not seen as "forced flags" per se.

They are forced for the installed ID. They aren't necessarily forced
for other different IDs that can be installed to replace the currently
installed ID.

The foo/bar-2.0 that you have installed is a completely different ID to
any foo/bar-2.0 you happen to have in a repository somewhere.

>
> >>> - get_masking_reason. Is the return value supposed to be used by
> >>>   anything other than a human?
> >> I doubt it.
> >
> > Specify that.
>
> I guess I'm again missing something ;). But I can't see, why it
> matters if it is intended for human readers or not :). And if there
> _is_ a difference, there should be no problem in adding a
> "for_humans" boolean argument to this call. :)
> Though I don't know, what the masking reason should look like for
> non-human targets.

If it's intended to be machine usable, the format has to be fixed.

>
> >>> - get_overlay_path. What if it's in an overlay with no on-disk
> >>> location? What if it's in some format the end tool doesn't
> >>> recognise?
> >> The end tool's problem I'd say.
> >
> > You're providing API calls that make this sort of thing difficult.
> > Not an ideal situation...
>
> Just return some kind of an URL. The end tool has to deal with it.

Then the name needs changing, and it needs to be documented that the
return value is entirely meaningless.

> >>> How does the whole privs thing work? Should anyone who can talk by
> >>> DBus be allowed to perform privileged operations? Should anyone
> >>> who can't perform privileged operations be allowed to do
> >>> unprivileged operations?
> >> I thought of having Catapult being a passive thing. So there is no
> >> way of doing privileged actions.
> >
> > Things like querying an uncached package are privileged.
>
> Yes? - Why? Any user can open an ebuild and have a look at the
> internals.

But they can't write to cache on disk. Remember that Portage needs you
to be in the 'portage' group to do anything, and anyone in the
'portage' group can easily obtain root on the box.

> >>> What about persistence? What's classed as a session? Why force it
> >>> down to a single session at once?
> >> Easier to implement. But it isn't forced. If the provider
> >> implements sessions (e.g. using the Dbus-sending-adress as a key)
> >> w/o extending API, I can't see a problem.
> >
> > But without well-defined session management, you're stuck when it
> > comes to multiple configurations and the like.
>
> What exactly do you want to store in / link to the session?

Which configuration set is active, for one...

-- 
Ciaran McCreesh