1 |
Hi everyone, |
2 |
|
3 |
some of you already know that work on GSoC project "Tree-wide collision |
4 |
checking and provided files database" has been started a few weeks ago. |
5 |
For the rest, I will make a short introduction and goals of this |
6 |
project (collagen). |
7 |
|
8 |
Collagen aims to improve quality of ebuilds in portage tree. It does |
9 |
this by compiling as many ebuilds as possible. It specifically takes |
10 |
into account various atoms in DEPEND variable. For example if package |
11 |
ebuild states that it needs =dev-libs/glib-2*, that package should be |
12 |
compilable with every version of glib-2* in portage (taking into account |
13 |
keywords). Therefore collagen will install one version of glib-2*, then |
14 |
ebuild in question, collect information, uninstall ebuild and first |
15 |
glib version. If repeats this process for every glib-2* in the tree. |
16 |
|
17 |
Original idea was to have two sides: |
18 |
* master server (matchbox) |
19 |
* slaves compiling packages (tinderboxes) |
20 |
|
21 |
Master server decides what needs to be compiled (automatically or |
22 |
semi-automatically). Tinderbox asks for job, master provides package |
23 |
name (and optionally version). Tinderbox then goes and tries to compile |
24 |
package with different sets of dependencies reporting results to |
25 |
Matchbox. |
26 |
|
27 |
It seems that whole process could be sped up by hosting binary |
28 |
packages on one central server (Binary host). Obviously various versions |
29 |
of the same package would be created and therefore unique names could be |
30 |
created by using some metadata to create hash part of filename. On a |
31 |
first thought I would use USE flags and DEPEND as metadata to hash. |
32 |
|
33 |
So far two other projects came to light as possible source of |
34 |
inspiration and/or collaboration: |
35 |
* catalyst (mainly tinderbox generating part) |
36 |
* AutotuA (automatic generic job framework) |
37 |
|
38 |
Especially AutotuA seems like good candidate for merging. |
39 |
|
40 |
It doesn't seem possible to compile every project with every version of |
41 |
every dependency, therefore I'd like to ask for your opinion especially |
42 |
about this part. One idea I had was to restrict testing to highest build |
43 |
number for given version. For example we have: |
44 |
glib-2.18.4-r1 and glib-2.18.4-r2, therefore we will only test against |
45 |
glib-2.18.4-r2 and will assume that r1 would be OK too (or users would |
46 |
upgrade since it's a bugfix release) |
47 |
|
48 |
Another approach to optimizing use of resources would be to have a |
49 |
priority list of packages that need most testing. I imagine this could |
50 |
be created by analyzing logs from gentoo mirrors, and figuring out which |
51 |
packages are downloaded most frequently. |
52 |
|
53 |
We would probably need at least one tinderbox per glibc version if I am |
54 |
not mistaken since this cannot be freely up/downgraded. |
55 |
|
56 |
|
57 |
This email was meant just as a teaser, more information (data model, UML |
58 |
diagrams) is available on project website (look for Documents): |
59 |
http://soc.gentooexperimental.org/projects/show/collision-database |
60 |
|
61 |
I'd love to be hear some suggestions, opinions and criticism. You can |
62 |
use this thread, or even various options on gentooexperimental.org. |
63 |
|
64 |
-- |
65 |
Stanislav Ochotnicky |
66 |
Working for Gentoo Linux http://www.gentoo.org |
67 |
Implementing Tree-wide collision checking and provided files database |
68 |
http://soc.gentooexperimental.org/projects/show/collision-database |
69 |
Blog: http://inputvalidation.blogspot.com/search/label/gsoc |
70 |
|
71 |
|
72 |
jabber: sochotnicky@×××××.com |
73 |
icq: 74274152 |
74 |
PGP: https://dl.getdropbox.com/u/165616/sochotnicky-key.asc |