Gentoo Logo
Gentoo Spaceship

Note: Due to technical difficulties, the Archives are currently not up to date. GMANE provides an alternative service for most mailing lists.
c.f. bug 424647
List Archive: gentoo-performance
Lists: gentoo-performance: < Prev By Thread Next > < Prev By Date Next >
To: gentoo-performance@g.o
From: Alec Warner <warnera6@...>
Subject: Re: gentoo-performance
Date: Fri, 20 Jan 2006 01:35:49 -0500
Hash: SHA1

Jeremy Brake wrote:
> How about speeding up the wait time on updating the portage cache after
> a sync.. even on my AMD 64 3500 it takes a number of minutes to chug
> through..
> are there any known ways to "vrrmmm" this up a little?
> Jeremy

Well the current problem is that in the 2.0 portage branch the cache
code sucks.  This is fixed in ~arch portage ( the 2.1_pre series ).  For
you users that don't want to upgrade to unstable, you should be able to
use cdb to speed up the process.

Setting RSYNC_EXCLUDES will not speed up the second half of the --sync (
the --metadata portion ).

Explanation:  The Portage Tree has a ton of ebuilds in it, when you do
something like emerge -pv <blar> portage used to have to go read each
ebuild and be like "oh what are the USE flags, the dependencies, etc.."
This takes a lot of time, especially for something like emerge -uD world
that may touch thousands of packages.

So to mitigate this issue portage has a "metadata cache" where it stores
the ebuild metadata so it doesn't have to source each ebuild every time.
 This gives performance increases ( reportedly ) of 100x speed-ups.

However, generating the metadata is time consuming, even on decent
hardware it will probably take 30-45 minutes.  To not piss the users
off, Gentoo calculates this metadata serve-side and serves it to you
during sync.

The Rsync portion of emerge --sync is pretty much everything before the
"updating metadata cache".  This should be pretty standard as far as
time goes on all boxes.  However the second portion is where portage has
to take the generated server-side cache and kind of "meld" it into your
already existant local cache ( stored in /var/cache/edb/dep for those of
you whom are curious ).  This didn't used to take a bunch of time..until
 KDE split their packages from KDE-monolith to KDE-meta.  The
switch probably won't help much in this area either.  Particularly the
slowdown at 50-52% is due to this portion of the tree.

There is a new cache system in the 2.1 branch of portage, or using cdb,
or perhaps even an sqlite back-end (patches and modules are out there),
will help until such time as the 2.1 branch is stablized ( probably not
too soon ).

- -Alec Warner (antarus)

Disclaimer: I'm not a developer (yet) but I've worked with the portage
team for some time.  Release dates, features, and other things are not
gauranteed to be correct just because I said so :)
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Using GnuPG with Thunderbird -

gentoo-performance@g.o mailing list

Re: gentoo-performance (sync speedups)
-- lnxg33k
Re: gentoo-performance
-- Jeremy Brake
-- Ken Robbins
Re: gentoo-performance
-- lnxg33k
Re: gentoo-performance
-- Chris
Re: gentoo-performance
-- lnxg33k
Re: gentoo-performance
-- Christopher Bergström
Re: gentoo-performance
-- Jeremy Brake
Lists: gentoo-performance: < Prev By Thread Next > < Prev By Date Next >
Previous by thread:
Re: gentoo-performance
Next by thread:
Re: gentoo-performance
Previous by date:
Re: gentoo-performance
Next by date:
Re: gentoo-performance

Updated Jun 17, 2009

Summary: Archive of the gentoo-performance mailing list.

Donate to support our development efforts.

Copyright 2001-2013 Gentoo Foundation, Inc. Questions, Comments? Contact us.