Re: [gentoo-guis] Why I don't like catapult

[René 'Necoro' Neumann]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ciaran McCreesh schrieb:
> On Sun, 30 Mar 2008 19:30:24 +0200
> René 'Necoro' Neumann <lists@××××××.eu> wrote:
>> Ciaran McCreesh schrieb:
>>> * use of cpv throughout. cpv isn't enough to uniquely identify an
>>> ID, especially if there're overlays. cpvr is closer, but still
>>> doesn't work if you need to deal with virtuals directly (rather
>>> than the thing provided by the virtual)
>> CPVR sounds reasonable ... and what about using virtual/* for
>> virtuals? Btw: There was the idea of providing a "get_virtuals"
>> function....
>
> The issue with virtuals is that all the package managers do them a bit
> differently. For Paludis, if you have two different providers of the
> same virtual installed with the same version and slot, you'll get two
> IDs that have the same cpvr and that are only distinguished by the
> package for which they are a virtual. This isn't an issue for us since
> we use a different PackageID instance for each, but if instead you're
> using a string to identify things (eww) you're in trouble.

Hmmm - ok then virtuals really need to be thought about.

>>> * use of regexes throughout. Every language defines its own,
>>> different regex dialect. There's no portability between languages.
>> True - I thought of using some kind of globbing - thus only allowing
>> "*" and "?" (and perhaps "^" and "$")
>
> Why support it at all? That's not something that should be in at the
> package manager level.

That's true ... but catapult is a bit higher than package manager level.
And at least the "find_" functions should allow them, to make searching
and alike simpler.

>>> * No mention of EAPI anywhere.
>> Why would this be needed? This is only true if "catapult API" ==
>> "EAPI".
>
> Well no. When doing pretty much anything with IDs, you have to be aware
> of the EAPI. Client programs mustn't attempt to do anything with IDs
> whose EAPI they don't recognise.

Hmm ... no. I see you are coming from the package-manager theory side.
And there EAPI is important. But I can't see any point here where EAPI
plays a role (besides package-IDs (which can be resolved using catapult
API versions)), that can't be covered _inside_ the provider.
I don't doubt that e.g. the catapult-paludis provider has to deal with
EAPI in some way. But the _user_ (or better: application dev) using
catapult API does not need to be aware of it. Catapult is of higher
level than package management itself.

>>> * Generally lots of hard-coded ebuildy things that don't map well
>>> onto future EAPIs or handling of non-ebuilds (e.g. native CRAN
>>> support).
>> For me - gentoo package management is all about ebuilds. So I can't
>> see, what should changed there.
>
> Future-direction-wise, a lot of people are wanting to handle things
> like CPAN, CRAN, Gems etc natively in the package manager rather than
> by writing wrapper ebuilds.

Ok. Perhaps we can postpone this until it gets real. (And then perhaps
use scpvr's ;) ... like: cpan:bla-foo/SomePerlPkg-0.13)

>>> * It's tied onto a single user configuration setup, and has no sane
>>> way of handling multiple configurations or unconfigured operation.
>> I don't understand this ...
>
> Paludis supports multiple configurations. You can, for example,
> have /etc/paludis, /etc/paludis-mychroot and ~/.paludis-anotherchroot,
> and Paludis can operate with all of thse. Paludis can also operate on
> systems where there's no configuration -- for example, you shouldn't
> need a full config setup just to be able to run QA checks on one
> particular repository.
>
> Portage also sort of has some of this in a very crude way, through
> PORTAGE_CONFIGROOT.

As catapult only queries, this can be resolved _inside_ the paludis
provider.
This of course gets hairy if it comes to libcatapult to provide
config-file manipulation...

>>> * sets don't map on to package manager sets.
>> This can easily be changed. But nevertheless, some catapult-wide sets
>> should be defined, that map onto the specific ones in the package
>> managers (e.g. catapult.SET_ALL should be replaced by the specific set
>> in the different managers)
>
> I'm not really sure that your concept of sets is right...
>

A "set" is a "set of packages" for me. You mention a "set of specs" -
don't get what kind of "specs" you are referring to here.

>>> - find_best. What does this do if you give it foo/bar-1 foo/baz-1?
>> Undefined atm.
>>>   What's the point of having this function at all?
>> See very first remark.
>
> Sure, but you should still define your API in such a way that it's
> easily generalisable and maps well onto the underlying data.
>

Hmm ... Catapult should provide an API to ease developing applications
that deal with package managers. So there might be the case here or
there that one provider has to do some work to map to the underlying
package manager.

>>> - find_packages. As find_best_match for search key. What
>>> disambiguation is performed upon the atom? What happens if you pass
>>> it something like 'git', which is ambiguous?
>> Return everything that fits. It should be on the users side to
>> translate it into something useful.
>
> That's messy.
>

Why? - It's a search function. And I guess in most cases the "return
everything that matches" is in case intended.

>>> - get_config_path. Unportable. What if the user is using some other
>>>   config format?
>> But the configs (read: package.mask, package.use etc.) have to be at a
>> specified unique location, don't they?
>
> For Paludis? No. NoConfigEnvironment, for example, uses no
> configuration directory at all.
>

This really is an issue... And what is happening if the user wants to
set an option (e.g. a useflag) in the "NoConfigEnvironment"? - Is a
config directory created and the environment switched?

>>> - get_deep_option. Deep's a mess; making package managers emulate
>>> it via a command line switch is silly.
>> If you don't have an equivalent: Return "".
>
> It shouldn't be in the API at all though.

Hmmm ... it is of course portage-inspired. So you are of course right,
that something like this should not be allowed in the API. Nevertheless,
I want to provide applications with the ability to run a package manager
with different option switches which follow some global semantics.
Or do you think, something like this is just not possible and we have to
look for another way of doing so?

>>> - get_merge_command. There's no nice way of using whatever this
>>> returns. And it's pointless -- why not just ask the package manager
>>> to do the merge?
>> Because catapult should be passive: I.E. only provide information and
>> does not do anything that could be harmful.
>
> Then it shouldn't even try to provide a way to do the merge stuff. It
> needs to either do this properly or not at all.

Think of the following use case: An application provides an interface
that allows users to select a package for installation. Now this
application uses catapult and wants to query for the command which has
to be run to get this package installed. Using get_merge_command, the
app could do:

spawn(catapult.system.get_merge_command(), package_list)

Catapult isn't the system merging here - it just provides the
information to actually _do_ the merge.

>>> - split_cpv. Why the weird revision thing?
>> What's so weired about it?
>
> Why is the revision considered special?

I'm waiting what's the result of the "-r0"-thread on gentoo-dev before
giving an anwer ;)

>>> - What do all of these do on unknown EAPI?
>> Again: Why does the API have to care about EAPI? - It's the providers
>> job to care about it.
>
> If you perform an API call that works on an ID with an unknown EAPI,
> what's supposed to happen? What about if you perform an API call that
> works upon an ID with an EAPI known to the package manager but not to
> the tool?
>
> (One example: what happens if we go with DEPENDENCIES for EAPI 2, and if
> you ask for DEPEND on an EAPI 2 package?)

We have the following cases:
1.) "DEPEND" is a fixed catapult term/constant. Then the provider has to
map to "DEPENDENCIES" if it encounters the term and EAPI2 is used.
2.) "DEPEND" is directly passed to the package manager (for example
using "get_package_settings"). I think this isn't preferred behavior and
only a given set of constants for the "settings" and alike should be
used. -> see 1.)

>>> - compare_version. But it takes packages. What's the ordering on
>>>   foo/bar-1 vs bar/baz-1?
>> Currently it would return: ("bar-1", "baz-1") as bar < baz. But this
>> can easily specified different. And a good point to think about error
>> handling again.
>
> Why the ordering on PN rather than CATEGORY/PN?

Hehe ;) - my fault... misread it. Of course it is first sorted using the
category. But haven't seen, they are different.

>>> And what's the return value? What about when || is in operation?
>> WWPMD (what would the package manager do): Simplified: Pass the
>> dep-string to the package manager and see what he does with it. The
>> result is returned.
>
> The package manager has to have special handling for || throughout. The
> behaviour of || blocks is rather tricky.

I guess it is a major problem if the package manager isn't able to
handle "||", right? - So I can't see your point here.

>>> - get_iuse_flags. What does this do for IUSE defaults? For installed
>>>   packages, all flags are forced.
>> defaults are returned verbatim. I used to define "forced flags" as:
>> "The flags the user can't change." (like userland_GNU or use.mask'ed
>> flags)
>
> The user can't change any flags for installed packages.

True. But he can re-install it using the changed flags. (Though he isn't
using the flags of the installed packages here, but the ones of the
uninstalled and overwriting the installed one.)
In other words: For this use case, the useflags of installed packages
are not seen as "forced flags" per se.

>>> - get_masking_reason. Is the return value supposed to be used by
>>>   anything other than a human?
>> I doubt it.
>
> Specify that.

I guess I'm again missing something ;). But I can't see, why it matters
if it is intended for human readers or not :). And if there _is_ a
difference, there should be no problem in adding a "for_humans" boolean
argument to this call. :)
Though I don't know, what the masking reason should look like for
non-human targets.

>>> - get_overlay_path. What if it's in an overlay with no on-disk
>>> location? What if it's in some format the end tool doesn't
>>> recognise?
>> The end tool's problem I'd say.
>
> You're providing API calls that make this sort of thing difficult. Not
> an ideal situation...

Just return some kind of an URL. The end tool has to deal with it.

>>> or when expanded keys aren't known.
>> When should that happen?
>
> Installed packages. USE_EXPAND wasn't (and still isn't by some package
> managers iirc) saved to VDB.

Ok - so you are trying to cover the case, that USE_EXPAND changes after
installation of a package.
My answer would be: This is the fault of the underlying package manager.
If USE_EXPAND can't be resolved correctly, the wrong result (what ever
that means in the special case) is passed back.

Catapult should try to get rid of some flaws - but if it is not
possible, we can't do anything here. Bad luck.

>>> What's the point in using DBus? What does it add over a simplified
>>> library with bindings?
>> Library bindings have several disadvantages:
>> - - Providers have to be coded in C.
>> - - How to access portage internals from C-Code? Yes: You can
>> implement python-calls in C... but I think this is messy.
>
> You can do the translation either way.

Right - but I consider it messy. (And I personally am a Python fan boy
and really dislike to code in C or even C++).
And I think it is quite strange, if you have a Python app, that accesses
Catapult-Portage:

Python (app) <---> Python (catapult bindings) <---> C (catapult) <--->
Python (portage)

>>> How does the whole privs thing work? Should anyone who can talk by
>>> DBus be allowed to perform privileged operations? Should anyone who
>>> can't perform privileged operations be allowed to do unprivileged
>>> operations?
>> I thought of having Catapult being a passive thing. So there is no way
>> of doing privileged actions.
>
> Things like querying an uncached package are privileged.

Yes? - Why? Any user can open an ebuild and have a look at the internals.

And if it still matters: DBus allows to define in config files, which
users are allowed to do which actions on which bus.

>>> What about persistence? What's classed as a session? Why force it
>>> down to a single session at once?
>> Easier to implement. But it isn't forced. If the provider implements
>> sessions (e.g. using the Dbus-sending-adress as a key) w/o extending
>> API, I can't see a problem.
>
> But without well-defined session management, you're stuck when it comes
> to multiple configurations and the like.

What exactly do you want to store in / link to the session?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFH8BYr4UOg/zhYFuARAlLnAJ0fdoxjIa3tj3vOz8iMjUDOgLD7cACghPk5
AnSwpVHV1KRy0txxDGHJ0a4=
=XoHx
-----END PGP SIGNATURE-----
-- 
gentoo-guis@l.g.o mailing list