Gentoo Archives: gentoo-dev

From: Auke Booij <auke@××××××.com>
To: gentoo-dev@l.g.o
Subject: [gentoo-dev] Package managers and new repositories formats
Date: Sat, 08 May 2010 22:32:57
1 Hey,
3 As part of Google Summer of Code, together with two other students and
4 our mentors, I'm looking into supporting non-ebuild repository formats
5 in existing package managers. Currently, in Portage these are
6 supported by externally generating ebuilds from available metadata in
7 advance, and then using these in Portage by putting them in a local
8 overlay (see g-cpan for an example). In Paludis, there is some
9 "native" support for non-ebuild repository types (they have exheres,
10 just for starters), and I've been told it's fairly trivial to add new
11 repository types in Pkgcore, too, simply by inheriting from a base
12 class.
14 My point is of course not to bash on Portage, but to come up with an
15 elegant solution to support a list of new repository types.
16 Personally, Google is about to pay me for adding support for R package
17 repositories (CRAN and Bioconductor), and the two other students will
18 be doing Pypi and PEAR support, respectively. We aren't looking
19 forward to doing one task nine times: support for package managers.
20 See, our projects communicate with two others: on one side, we have to
21 read and interpret package repositories, on the other side, we need to
22 communicate with all the different package managers. Reading
23 repositories is something we'll only have to implement once for a
24 given repository format, but passing the relevant data to portage,
25 pkgcore or paludis is something less trivial.
27 We, students and mentors, have thought of various plans to only have
28 to write repository "plugins" and tackle the package manager side
29 together once and for all, but we couldn't reach agreement. How can we
30 support the three existing package managers, and any future package
31 manager which only supports PMS? To accomplish PMS-only package
32 manager support, the repository code would at least have to be able to
33 generate ebuilds, but perhaps we could come up with something better
34 than simply translating one type of metadata into the other, for
35 slightly more advanced package managers? Could we perhaps come up with
36 a *standard* for non-ebuild repository type /definitions/? Ideally,
37 developers would only have to write one chunk of code to read a
38 repository format, and then all package managers would be able to read
39 repositories of that type.
41 Now, before we roll in the discussion of stability: yes, blindly
42 importing packages from upstream is dangerous, and yes, it may just
43 set your cat on fire. However, it's a matter of fact that the package
44 maintainers simply don't have the time to manually check all packages,
45 and isn't Gentoo all /about/ at least having /access/ to those nuclear
46 plants? If this monster turns out to be deadly, we can always disable
47 support by default. That said, I do believe many packages, especially
48 in CRAN and Bioconductor, eventually will not do much more harm than
49 cause inflation of the U.S. dollar, which honestly cannot be helped
50 anyway. Wait, did I say Bioconductor? Oh, then terrorists with access
51 to Gentoo may have less trouble developing their own biological
52 weapons, but so be it, as long as they publish their DNA code as
53 GPL...
55 The final questions I'm really here for, then, are: how do you feel
56 about standardizing repository format definitions, how should we
57 support new repository types in current package managers, how should
58 we go about constructing a common interface for new format definitions
59 and why is it always on days like these I run out of coffee?
61 Thanks for all your thoughts!
62 Auke Booij / tulcod.