Gentoo Archives: gentoo-dev

From: Corentin Chary <corentin.chary@×××××.com>
To: "Michał Górny" <mgorny@g.o>
Cc: gentoo-dev@l.g.o
Subject: Re: [gentoo-dev] euscan proof of concept (like debian's uscan)
Date: Fri, 23 Sep 2011 09:57:06
In Reply to: Re: [gentoo-dev] euscan proof of concept (like debian's uscan) by "Michał Górny"
On Mon, Sep 19, 2011 at 10:53 AM, Michał Górny <mgorny@g.o> wrote:
> On Mon, 19 Sep 2011 10:39:11 +0200 > Corentin Chary <corentin.chary@×××××.com> wrote: > >> ## Also update eix database, because we use eix internaly >> ## Bottleneck: disk and cpu >> ##Time: 30mn ~ 1h >> eix-update > > Using egencache to keep caches for overlays will make eix updates much > faster. > > Here's my code for it (it uses overlays in /usr/portage/local): > > cd /usr/portage/local && \ > for O in */; do >        echo "${O}" >        egencache --jobs=8 --update --update-use-local-desc --rsync \ >                --repo=$(cat ${O}profiles/repo_name) > done >
Just done that, and indeed, it's much faster. Also I switched to PostgreSQL, and since that avoids a lot of deadlock situations when making queries in parallel, I'm now able to do that: eix --only-names -x | gparallel --eta --load 8 --jobs 400% --max-args=64 python scan-metadata And now it takes less than 30mn instead of more than 1h, and more importantly, it scales. -- Corentin Chary