Gentoo Archives: gentoo-portage-dev

From: Brian Dolbec <dolsen@g.o>
To: gentoo-portage-dev <gentoo-portage-dev@l.g.o>
Subject: [gentoo-portage-dev] New proposed modular sync system
Date: Thu, 09 Jan 2014 17:37:51
1 First off, I know many of you think portage needs to do everything on
2 it's own. NO outside dependencies.
4 BUT! Please hear me out.
6 While working on many of gentoo's tools gentoolkit's equery, eclean,
7 enalyze, Layman, mirrorselect, etc.. I have found that there was a lot
8 of needless code duplication. Both within the apps themselves and
9 copying/re-inventing the nearly identical code in other apps.
11 I want to move many of gentoo's core applications to using some more
12 shared libraries. But that in itself is not a topic to discuss in this
13 thread. Please, do not sidetrack this thread. The above is only for
14 background info leading to this proposed change. This proposal follows
15 along that thinking.
17 Many years ago after re-writing ecleans code I began re-writing
18 portage's emaint code so that it could be imported and used internally
19 in eclean to re-generate the Packages binpkg index file. In that code I
20 found a todo which said "make a plug-in system so we don't have to keep
21 modifying this code" or something along those lines. So I did. It
22 took 4 years+, rebasing many times and migrating vcs's, but Zac finally
23 merged it. It has been used in portage for more than a year. That
24 plug-in system I designed to be very flexible and basic. It can be used
25 for many areas of portage to simplify it's code and the maintenance
26 required to keep current with changes in repository types, and transfer
27 methods.
29 I have started a Proposals sub-page under the Portage project page in
30 the wiki. It has a link to a diagram I made showing how the plug-in
31 system is laid out. This thread will be used to discuss the proposal
32 and the details needed for the module_spec dictionary that is used to
33 provide the manager with the details of what a module provides, options,
34 etc..
36 For a working example of the system in action, just try out emaint -h,
37 and some other actions it provides. Then move one of it's modules out
38 of the the emaint/modules/ directory and re-try emaint -h. You will see
39 the options for that module and it's actions are no longer available.
40 No code editing was required, no errors occurred,... Put it back,
41 retry...
43 The modular sync system I propose would re-use that plug-in manager to
44 interface the sync manager code with the modules that perform the sync
45 actions. This sytem is easily extended with new modules for different
46 transfer methods or VCS specific modules.
48 This also makes it possible for other gentoo applications like layman to
49 install a layman specific module which syncs all layman overlays that it
50 installs. All this done with out effort on the part of the portage team
51 either for maintenance or code changes to support it's addition. The
52 same can be said for Funtoo, if our git module is not suiotable, they
53 just install their own custom sync module. Done :)
55 These add-on sync modules can also be installed via their own ebuild.
57 For example: Zac has for years had a custom script that syncs the
58 portage tree and makes a squashfs version of it which he uses for his
59 PORTDIR. it is faster than a normal one. He, or someone else could make
60 a app-portage/squashfs-sync ebuild that installs a module for that
61 purpose. The user then just needs to edit his repos.conf file and
62 change the sync-type to squashfs. Then emerge --sync will do it
63 automatically without user intervention and auxiliary scripts run
64 manually or cron job.
66 This system would also make it possible to modify emerge-s cli to accept
67 target repos to sync. and not any others also installed. Similarly to
68 "layman -s some-overlay", "emerge --sync some-overlay"
70 P.S. sorry for such a long email
71 --
72 Brian Dolbec <dolsen@g.o>


File name MIME type
signature.asc application/pgp-signature