[gentoo-user] Portage 50-51% and emerge metadata timings

[Peter]

peter@mars ~ $ sudo emerge info
Portage 2.0.53 (default-linux/x86/2005.0, gcc-3.4.4, glibc-2.3.5-r2, 2.6.14-gentoo-r5 i686)
================================================================= 
System uname: 2.6.14-gentoo-r5 i686 AMD Athlon(tm) XP 2800+ 
Gentoo Base System version 1.6.14
dev-lang/python:     2.4.2
sys-apps/sandbox:    1.2.12
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1
sys-devel/binutils:  2.16.1
sys-devel/libtool:   1.5.20
virtual/os-headers:  2.6.11-r3

I'd seen all the threads on the problems with portage getting held up
during emerge metadata. Since around a month ago, I have suffered through
the holdup almost every day. Sometimes, it would take 5 or more minutes!

I read as much as I could on the various forums and ng and wanted to post
MY suggested workaround.

In a nutshell, I found that portage needed to be on its own partition and
had to be backed up and restored onto a clean partition. By doing this
alone, there was no more dreaded 50-51% holdup.

I also found that it did not make much difference whether I used ext2,3 or
reiserfs when running emerge metadata. In fact, reiser was slower by
around 30 seconds (57-ext3 vs 1:30-reiser). This was not surprising since
on the namesys website they note that programs like find won't perform as
well due to its internal organization. Interestingly, if I ran emerge
metadata a second time on reiser, the speed dropped to 45 seconds. I
imagine this is because of directory caching.

On my particular setup, I have one partition that includes portage,
portage overlay, distfiles, and portage tmp dir. I set all these in
make.conf.

Having portage in /usr is kind of a contradiction imho, since most files
there are supposed to be static, not changing all the time. Moving portage
and its related files elsewhere makes everything a log cleaner, and at
least in my case, faster.

>From my make.conf. Note, /mnt/src is my portage partition.

PORT_LOGDIR="/var/log/portage"
PORTDIR="/mnt/src/portage"
PORTDIR_OVERLAY="/mnt/src/local/portage"
PORTAGE_TMPDIR="/mnt/src/var/tmp"
DISTDIR="/mnt/src/distfiles"

For right now, I have reiser handling /mnt/src, and I'll leave this alone
for a while. Don't know if performance will improve. It _might_ since
the act of syncing will force a read through the entire portage tree
which might make metadata faster

HTH.

Suggestions:

0) You _are_ backed up, right? At least, save the directories you'll be
working on!
1) make a partition. 1+G for /usr/portage. Larger for additional files. 
I have 7 gig for my setup. Filesystem is up to you. Just make sure kernel
support is enabled (i.e. reiser) and appropriate support files loaded.
2) copy over existing portage using cp -Rp (preserve times and
permissions otherwise portage will download the entire tree), or tar it
and untar it. Whatever works. Just make sure that make.conf is adjusted
and /etc/make.profile is relinked to the new location. 

/etc/make.profile -> /mnt/src/portage/profiles/default-linux/x86/2005.0

This is important otherwise any emerge command will bomb. Your link may
differ depending on your arch. ls -l /etc/make.profile will show what you
need to do. Just substitute your new portage location and add the rest.

3) update make.conf. 
4) see if everything works, try emerge -s, emerge sync, metadata.
5) if it does, you can purge /usr/portage and any other directories you
moved over to your dedicated portage partition.


-- 
gentoo-user@g.o mailing list