Re: [gentoo-portage-dev] portage doc & cache

[John Nilsson]

On Fri, 2004-06-11 at 23:15, Philippe Lafoucrière wrote:
> Hi all
>
> I'm wondering where I can find some doc about portage, and especially
> the cache part. I use debian at work, and get so sad when doing an
> emerge rsync at home (10 or 15 minutes to rsync, and almost 5 minutes to
> calculate world dependancies) on a bi-athlon 1.8GHz.
>
> I just wanted to check that the cache system is really a cache :)
>
> maybe ebuilds should be "compiled" into seriaziled objects or something
> similar. Only an emerge rsync (or something called "emerge update")
> would fetch new ebuilds, and compile new ones (we don't care anymore of
> hand modified files after sync).

Mabey rfc2616 can be of some nice inspiration about caching.
As always, I refer to Roy T. Fielding's "Architectural Styles and
the Design of Network-based Software Architectures" as a must read.

>
>
> On the other hand, I was thinkink of the "rsync way" to synchronise the
> portage tree. Why don't we use a portage tree ID (incremented each time
> an ebuild is commited to CVS). For exemple, I have the tree number
> 12512. Current portage tree ID is 12514. I just need too fresh ebuilds.

You have just described subversion...
Anyway you might want to experiment with cvsup vs. rsync.

> Moreover, do I REALLY need all ebuilds content ???? Why can't portage
> just look at some headers (with arch, desc, etc.) and left the install
> part on other files. Rsync local tree would be lighter, and faster to
> sync. When you need to emerge some program, emerge would download the
> _real_ ebuild file, the digests, the distfiles, etc.

Do some experiments and benchmarking. What if only the cache was
downloaded at sync? Say you have a sql backend for portage, just sync
that and download the ebuilds on demand.


> The headers can also be contained in one large tar.gz file, to reduce
> bandwith again (I know, like debian).
>
> I know this stuff might let you think I'm crazy to rebuild all portage,
> but here are some ideas to dig :D

Personally I like crazy =) I think you should have a better
understanding of the current system before you criticise it though. 

<quote>rsync is a file transfer program for Unix systems. rsync uses the
"rsync algorithm" which provides a very fast method for bringing remote
files into sync. It does this by sending just the differences in the
files across the link, without requiring that both sets of files are
present at one of the ends of the link beforehand.</quote>

rsync allready does it's best to only download new information.

-John