Gentoo Logo
Gentoo Spaceship




Note: Due to technical difficulties, the Archives are currently not up to date. GMANE provides an alternative service for most mailing lists.
c.f. bug 424647
List Archive: gentoo-performance
Navigation:
Lists: gentoo-performance: < Prev By Thread Next > < Prev By Date Next >
Headers:
To: gentoo-performance@g.o
From: Alec Warner <warnera6@...>
Subject: Re: gentoo-performance
Date: Fri, 20 Jan 2006 01:35:49 -0500
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jeremy Brake wrote:
> How about speeding up the wait time on updating the portage cache after
> a sync.. even on my AMD 64 3500 it takes a number of minutes to chug
> through..
> are there any known ways to "vrrmmm" this up a little?
> 
> Jeremy

Well the current problem is that in the 2.0 portage branch the cache
code sucks.  This is fixed in ~arch portage ( the 2.1_pre series ).  For
you users that don't want to upgrade to unstable, you should be able to
use cdb to speed up the process.

Setting RSYNC_EXCLUDES will not speed up the second half of the --sync (
the --metadata portion ).

Explanation:  The Portage Tree has a ton of ebuilds in it, when you do
something like emerge -pv <blar> portage used to have to go read each
ebuild and be like "oh what are the USE flags, the dependencies, etc.."
This takes a lot of time, especially for something like emerge -uD world
that may touch thousands of packages.

So to mitigate this issue portage has a "metadata cache" where it stores
the ebuild metadata so it doesn't have to source each ebuild every time.
 This gives performance increases ( reportedly ) of 100x speed-ups.

However, generating the metadata is time consuming, even on decent
hardware it will probably take 30-45 minutes.  To not piss the users
off, Gentoo calculates this metadata serve-side and serves it to you
during sync.

The Rsync portion of emerge --sync is pretty much everything before the
"updating metadata cache".  This should be pretty standard as far as
time goes on all boxes.  However the second portion is where portage has
to take the generated server-side cache and kind of "meld" it into your
already existant local cache ( stored in /var/cache/edb/dep for those of
you whom are curious ).  This didn't used to take a bunch of time..until
 KDE split their packages from KDE-monolith to KDE-meta.  The X.org
switch probably won't help much in this area either.  Particularly the
slowdown at 50-52% is due to this portion of the tree.

There is a new cache system in the 2.1 branch of portage, or using cdb,
or perhaps even an sqlite back-end (patches and modules are out there),
will help until such time as the 2.1 branch is stablized ( probably not
too soon ).

- -Alec Warner (antarus)

Disclaimer: I'm not a developer (yet) but I've worked with the portage
team for some time.  Release dates, features, and other things are not
gauranteed to be correct just because I said so :)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQ9CExWzglR5RwbyYAQLJEhAAj7i2cRVgv1iBfi61mGkn9q1t/K5JEJcv
zK07bTmMDirviessdKiZVSufXdWV79tW0MDEVqmGI9t+V+uwM77aFbge1VSkEaQB
RWetuQL8tQRUX1KvQ+ItScVtbqKIAQe/Jn+BwSim05jF3+fF15Z6EpPPKypNLQxK
2ef/bcJ91Gctv0xcd6j943uOPFDCt05Ahe06/pQ0wgGdnAcKLOy/RVwDOVfprXim
AwiVsU4aCxRI056RkEj1VuSwSYQEa91WKpTGv81lZVkRW8LRxkgc7UAydMYGjOY5
1prjv2Koyly5ycVvxshKVmLfuaqByY9bBnlklyKDdFW1ZD1udCflCLxmz3GuTCsm
/FHce+Y9smqq3sF1wV7DYXu9vTnLdBqVBlDq4cEtd5XdgQm3TJ5rUGF2Mepicyhg
Bg4ibDExDB5eKWCnGU3ioSCd8TY9cdR9ZXxpm6JLGfr9ztog0/vScsIZbj3dS4RH
WqGklvW0F9N6NjP26WJwtQsmSIZpSJU4yPxneZOdQxGOUfdNPQap+qO+rZaitPKW
yWaikaiSuecPSul0ithpGnCFttPHrBLyNKNl7aom04Bht4KSdaqzriIL/RylWzHW
OgazUV9RuVdI8oEx+q3zKzOgvG3dXeL7HNdhH+j9noIyGVa8ouNHWMGfFUc1baxV
7eyB1buSeQY=
=vWqe
-----END PGP SIGNATURE-----
-- 
gentoo-performance@g.o mailing list


Replies:
Re: gentoo-performance (sync speedups)
-- lnxg33k
Re: gentoo-performance
-- Jeremy Brake
References:
gentoo-performance
-- Ken Robbins
Re: gentoo-performance
-- lnxg33k
Re: gentoo-performance
-- Chris
Re: gentoo-performance
-- lnxg33k
Re: gentoo-performance
-- Christopher Bergström
Re: gentoo-performance
-- Jeremy Brake
Navigation:
Lists: gentoo-performance: < Prev By Thread Next > < Prev By Date Next >
Previous by thread:
Re: gentoo-performance
Next by thread:
Re: gentoo-performance
Previous by date:
Re: gentoo-performance
Next by date:
Re: gentoo-performance


Updated Jun 17, 2009

Summary: Archive of the gentoo-performance mailing list.

Donate to support our development efforts.

Copyright 2001-2013 Gentoo Foundation, Inc. Questions, Comments? Contact us.