| 1 |
On 03/08/2010 09:21 AM, Robert R. Russell wrote: |
| 2 |
> The cache sync project[1] wants a way to generate portage's cache on the portage tree and/or any chosen overlay and then distribute that cache by some method. Correct? |
| 3 |
|
| 4 |
Well, you can already do that with the egencache program that's |
| 5 |
included with portage. I think the gist of the "cache sync" idea is |
| 6 |
that you should be able to download the cache for dependency |
| 7 |
calculations, and defer the download of the source package until |
| 8 |
after the dependency calculation. For doing something like that, a |
| 9 |
portage tree is probably not very suitable since the tree can change |
| 10 |
rapidly and the cache may invalidate quickly. If the cache and the |
| 11 |
source package will be distributed separately, it might be more |
| 12 |
practical to make something like a source rpm that contains an |
| 13 |
ebuild and eclasses. Many of these source packages could be |
| 14 |
distributed in a repository that is independent of the portage tree, |
| 15 |
and it's cache may be valid for a longer period of time. |
| 16 |
|
| 17 |
> This project sounds very similar to an idea I have been toying around with for a bit, but I have some questions before I apply for this project. |
| 18 |
> |
| 19 |
> How well documented is the current cache format portage uses? |
| 20 |
|
| 21 |
It's not very well documented. You might try experimenting with the |
| 22 |
egencache program to get a feel for how it works. Cache is generated |
| 23 |
by sourcing ebuilds, and it's stored in /var/cache/edb/dep. It's |
| 24 |
validated by comparing ebuild and eclass timestamps to those that |
| 25 |
are saved in the cache entry. After a complete cache entry is |
| 26 |
generated for /var/cache/edb/dep, an incomplete cache entry (lacking |
| 27 |
eclass timestamps, since the format hasn't been extended to support |
| 28 |
them yet) is written into $PORTDIR/metadata/cache. There is a |
| 29 |
discussion about extending the format to include eclass digests here: |
| 30 |
|
| 31 |
http://archives.gentoo.org/gentoo-dev/msg_cfa80e33ee5fa6f854120ddfb9b468b3.xml |
| 32 |
|
| 33 |
> What restrictions if any would be placed on extending the current cache format? |
| 34 |
|
| 35 |
It has to be backward compatible. If we want to change the format in |
| 36 |
a backward incompatible way, for example by combining the whole |
| 37 |
cache into a single text file, we'll have to distribute both formats |
| 38 |
until users have had time to migrate to a package manager that |
| 39 |
supports the new format. |
| 40 |
|
| 41 |
> How well documented is the ebuild file format? |
| 42 |
|
| 43 |
It's pretty well documented by PMS. You can get that by installing |
| 44 |
app-doc/pms. For something that's much shorter and less |
| 45 |
comprehensive, there's the `man 5 ebuild`. |
| 46 |
|
| 47 |
> How much of the ebuild is essential for portage to create a valid cache entry? |
| 48 |
|
| 49 |
The whole ebuild and any eclasses that it inherits. |
| 50 |
|
| 51 |
> How stable and well documented is the format of the cache essential pieces of an ebuild? |
| 52 |
|
| 53 |
It's very stable because it has to be backward compatible. Breaking |
| 54 |
compatibility would be a sever problem because dependency |
| 55 |
calculations are very slow unless there is a valid/compatible cache |
| 56 |
available. |
| 57 |
|
| 58 |
> Is there any previous work on this or a project that might overlap with this project? Such as, an attempt at a new parser for portage. |
| 59 |
|
| 60 |
I know that Mounir Lamouri (volkmar@gentoo) has been thinking about |
| 61 |
a new cache format that will use a single file for the whole cache. |
| 62 |
|
| 63 |
> Will there be mandatory discussion between the person doing this project and the person doing the tags support project? |
| 64 |
|
| 65 |
Tags are a separate project. |
| 66 |
|
| 67 |
> Is improving the performance of the cache and/or search feature a mandatory goal of this project? |
| 68 |
|
| 69 |
Well, the cache should probably all go in a single file, and that |
| 70 |
will probably improve performance because generatlly it's faster to |
| 71 |
load one big file than a bunch of small files. |
| 72 |
|
| 73 |
> Thank you. |
| 74 |
> |
| 75 |
> [1] http://en.gentoo-wiki.com/wiki/Google_Summer_of_Code_2010_ideas#Cache_sync |
| 76 |
> |
| 77 |
-- |
| 78 |
Thanks, |
| 79 |
Zac |
Archives