| 1 |
Hi everyone, |
| 2 |
|
| 3 |
some of you already know that work on GSoC project "Tree-wide collision |
| 4 |
checking and provided files database" has been started a few weeks ago. |
| 5 |
For the rest, I will make a short introduction and goals of this |
| 6 |
project (collagen). |
| 7 |
|
| 8 |
Collagen aims to improve quality of ebuilds in portage tree. It does |
| 9 |
this by compiling as many ebuilds as possible. It specifically takes |
| 10 |
into account various atoms in DEPEND variable. For example if package |
| 11 |
ebuild states that it needs =dev-libs/glib-2*, that package should be |
| 12 |
compilable with every version of glib-2* in portage (taking into account |
| 13 |
keywords). Therefore collagen will install one version of glib-2*, then |
| 14 |
ebuild in question, collect information, uninstall ebuild and first |
| 15 |
glib version. If repeats this process for every glib-2* in the tree. |
| 16 |
|
| 17 |
Original idea was to have two sides: |
| 18 |
* master server (matchbox) |
| 19 |
* slaves compiling packages (tinderboxes) |
| 20 |
|
| 21 |
Master server decides what needs to be compiled (automatically or |
| 22 |
semi-automatically). Tinderbox asks for job, master provides package |
| 23 |
name (and optionally version). Tinderbox then goes and tries to compile |
| 24 |
package with different sets of dependencies reporting results to |
| 25 |
Matchbox. |
| 26 |
|
| 27 |
It seems that whole process could be sped up by hosting binary |
| 28 |
packages on one central server (Binary host). Obviously various versions |
| 29 |
of the same package would be created and therefore unique names could be |
| 30 |
created by using some metadata to create hash part of filename. On a |
| 31 |
first thought I would use USE flags and DEPEND as metadata to hash. |
| 32 |
|
| 33 |
So far two other projects came to light as possible source of |
| 34 |
inspiration and/or collaboration: |
| 35 |
* catalyst (mainly tinderbox generating part) |
| 36 |
* AutotuA (automatic generic job framework) |
| 37 |
|
| 38 |
Especially AutotuA seems like good candidate for merging. |
| 39 |
|
| 40 |
It doesn't seem possible to compile every project with every version of |
| 41 |
every dependency, therefore I'd like to ask for your opinion especially |
| 42 |
about this part. One idea I had was to restrict testing to highest build |
| 43 |
number for given version. For example we have: |
| 44 |
glib-2.18.4-r1 and glib-2.18.4-r2, therefore we will only test against |
| 45 |
glib-2.18.4-r2 and will assume that r1 would be OK too (or users would |
| 46 |
upgrade since it's a bugfix release) |
| 47 |
|
| 48 |
Another approach to optimizing use of resources would be to have a |
| 49 |
priority list of packages that need most testing. I imagine this could |
| 50 |
be created by analyzing logs from gentoo mirrors, and figuring out which |
| 51 |
packages are downloaded most frequently. |
| 52 |
|
| 53 |
We would probably need at least one tinderbox per glibc version if I am |
| 54 |
not mistaken since this cannot be freely up/downgraded. |
| 55 |
|
| 56 |
|
| 57 |
This email was meant just as a teaser, more information (data model, UML |
| 58 |
diagrams) is available on project website (look for Documents): |
| 59 |
http://soc.gentooexperimental.org/projects/show/collision-database |
| 60 |
|
| 61 |
I'd love to be hear some suggestions, opinions and criticism. You can |
| 62 |
use this thread, or even various options on gentooexperimental.org. |
| 63 |
|
| 64 |
-- |
| 65 |
Stanislav Ochotnicky |
| 66 |
Working for Gentoo Linux http://www.gentoo.org |
| 67 |
Implementing Tree-wide collision checking and provided files database |
| 68 |
http://soc.gentooexperimental.org/projects/show/collision-database |
| 69 |
Blog: http://inputvalidation.blogspot.com/search/label/gsoc |
| 70 |
|
| 71 |
|
| 72 |
jabber: sochotnicky@×××××.com |
| 73 |
icq: 74274152 |
| 74 |
PGP: https://dl.getdropbox.com/u/165616/sochotnicky-key.asc |
Archives