Gentoo Logo
Gentoo Spaceship

Note: Due to technical difficulties, the Archives are currently not up to date. GMANE provides an alternative service for most mailing lists.
c.f. bug 424647
List Archive: gentoo-qa
Lists: gentoo-qa: < Prev By Thread Next > < Prev By Date Next >
To: gentoo-qa@g.o
From: Stanislav Ochotnicky <sochotnicky@...>
Subject: [GSoC-status] Tree-wide collision checking and files database
Date: Fri, 12 Jun 2009 15:32:35 +0200
Hi everyone,

some of you already know that work on GSoC project "Tree-wide collision
checking and provided files database" has been started a few weeks ago.
For the rest, I will make a short introduction and goals of this
project (collagen).

Collagen aims to improve quality of ebuilds in portage tree. It does
this by compiling as many ebuilds as possible. It specifically takes
into account various atoms in DEPEND variable. For example if package
ebuild states that it needs =dev-libs/glib-2*, that package should be
compilable with every version of glib-2* in portage (taking into account
keywords). Therefore collagen will install one version of glib-2*, then
ebuild in question, collect information, uninstall ebuild and first
glib version. If repeats this process for every glib-2* in the tree.

Original idea was to have two sides:
 * master server (matchbox)
 * slaves compiling packages (tinderboxes)

Master server decides what needs to be compiled (automatically or
semi-automatically). Tinderbox asks for job, master provides package
name (and optionally version). Tinderbox then goes and tries to compile
package with different sets of dependencies reporting results to

It seems that whole process could be sped up by hosting binary
packages on one central server (Binary host). Obviously various versions
of the same package would be created and therefore unique names could be
created by using some metadata to create hash part of filename. On a
first thought I would use USE flags and DEPEND as metadata to hash.

So far two other projects came to light as possible source of
inspiration and/or collaboration:
 * catalyst (mainly tinderbox generating part)
 * AutotuA (automatic generic job framework)

Especially AutotuA seems like good candidate for merging.

It doesn't seem possible to compile every project with every version of
every dependency, therefore I'd like to ask for your opinion especially
about this part. One idea I had was to restrict testing to highest build
number for given version. For example we have:
glib-2.18.4-r1 and glib-2.18.4-r2, therefore we will only test against
glib-2.18.4-r2 and will assume that r1 would be OK too (or users would
upgrade since it's a bugfix release)

Another approach to optimizing use of resources would be to have a
priority list of packages that need most testing. I imagine this could
be created by analyzing logs from gentoo mirrors, and figuring out which
packages are downloaded most frequently. 

We would probably need at least one tinderbox per glibc version if I am
not mistaken since this cannot be freely up/downgraded. 

This email was meant just as a teaser, more information (data model, UML
diagrams) is available on project website (look for Documents):

I'd love to be hear some suggestions, opinions and criticism. You can
use this thread, or even various options on

Stanislav Ochotnicky
Working for Gentoo Linux
Implementing Tree-wide collision checking and provided files database

jabber: sochotnicky@...
icq: 74274152
pgpWbtQbG9ImQ.pgp (PGP signature)
Lists: gentoo-qa: < Prev By Thread Next > < Prev By Date Next >
Previous by thread:
QA status update
Next by thread:
Re: [gentoo-soc] [GSoC-status] Tree-wide collision checking and files database
Previous by date:
Re: QA status update
Next by date:
Re: [gentoo-soc] [GSoC-status] Tree-wide collision checking and files database

Updated Aug 15, 2009

Summary: Archive of the gentoo-qa mailing list.

Donate to support our development efforts.

Copyright 2001-2013 Gentoo Foundation, Inc. Questions, Comments? Contact us.