Re: [gentoo-catalyst] Binary package dependencies and update_seed

[W. Trevor King]

On Thu, Apr 11, 2013 at 04:34:12PM -0400, W. Trevor King wrote:
> On Thu, Apr 11, 2013 at 01:24:19PM -0700, Matt Turner wrote:
> > So, to make sure that I'm on the same page: is the the problem that
> > we're using stale packages in stage1 and if so, where did they come
> > from? A previous stage1 build that didn't do update-seed?
>
> That's where mine (and presumably iamben's) came from.  However, they
> could also (I think) come from a stage1 build that used an older
> snapshot.  I'm testing now with a build from:
>
>   subarch: i686
>   version_stamp: 2013.1
>   target: stage1
>   rel_type: default
>   profile: default/linux/x86/13.0/desktop
>   portage_confdir: /var/tmp/catalyst/portage-conf/default/
>   snapshot: 20130208
>   source_subpath: default/stage3-i686-20121213
>   update_seed: yes
>
> Followed by another build with the same version_stamp but using:
>
>   snapshot: 20130308
>
> The idea is that the first build might produce packages linking
> libmpc.so.2, and the second build might reuse those packages, despite
> the stabilization of mpc-1.0.1 in the tree.

So I finished the build and I have a broken stage1 after the tree
update (as I expected).  Here's how it goes:

Clear the caches:

  # rm -rf --one-file-system /tmp/*.log /var/tmp/catalyst/{kerncache,packages,tmp}

Confirm the first spec (using the older tree)

  # cat /var/tmp/catalyst/spec/default-stage1-i686-test.1.spec
  subarch: i686
  version_stamp: 2013.1
  target: stage1
  rel_type: default
  profile: default/linux/x86/13.0/desktop
  portage_confdir: /var/tmp/catalyst/portage-conf/default/
  snapshot: 20130208
  source_subpath: default/stage3-i686-20121213
  update_seed: yes

Build it:

  # python2.7 catalyst -c catalyst.conf -f /var/tmp/catalyst/spec/default-stage1-i686-test.1.spec 2>&1 | tee /tmp/stage1-i686-test.1.log
  …
  >>> Emerging (30 of 38) sys-devel/gcc-4.6.3 for /tmp/stage1root/
  >>> Installing (30 of 38) sys-devel/gcc-4.6.3 to /tmp/stage1root/
  …

As expected, with a tree where mpc-0.8.2 is the latest stable version,
we get dependencies on libmpc.so.2:

  # less /var/tmp/catalyst/tmp/default/stage1-i686-2013.1/tmp/stage1root/usr/libexec/gcc/i686-pc-linux-gnu/4.6.3/cc1 | grep libmpc
   0x00000001 (NEEDED)                     Shared library: [libmpc.so.2]
  # ls -l /var/tmp/catalyst/tmp/default/stage1-i686-2013.1/tmp/stage1root/usr/libexec/gcc/i686-pc-linux-gnu/4.6.3/cc1 
  -rwxr-xr-x 1 root root 10557140 Apr 11 21:24 /var/tmp/catalyst/tmp/default/stage1-i686-2013.1/tmp/stage1root/usr/libexec/gcc/i686-pc-linux-gnu/4.6.3/cc1
  # less /var/tmp/catalyst/packages/default/stage1-i686-2013.1/sys-devel/gcc-4.6.3.tbz2 | grep '/cc1$'
  -rwxr-xr-x root/root  10557140 2013-04-11 21:24 ./usr/libexec/gcc/i686-pc-linux-gnu/4.6.3/cc1

Let's shift that stage out of the way for safe keeping and build
another with a new snapshot:

  # mv /var/tmp/catalyst/tmp/default/stage1-i686-{2013.1,test.1}/
  # diff /var/tmp/catalyst/spec/default-stage1-i686-test.{1,2}.spec
  7c7
  < snapshot: 20130208
  ---
  > snapshot: 20130308
  # python2.7 catalyst -c catalyst.conf -f /var/tmp/catalyst/spec/default-stage1-i686-test.2.spec 2>&1 | tee /tmp/stage1-i686-test.2.log
  …
  >>> Emerging binary (10 of 75) sys-devel/gcc-config-1.7.3 for /tmp/stage1root/
  >>> Installing (10 of 75) sys-devel/gcc-config-1.7.3 to /tmp/stage1root/
  …

Looks like were using the out-of-date binary package.  Let's check to
see we're still linking against the old libmpc:

  # less /var/tmp/catalyst/tmp/default/stage1-i686-2013.1/tmp/stage1root/usr/libexec/gcc/i686-pc-linux-gnu/4.6.3/cc1 | grep libmpc
   0x00000001 (NEEDED)                     Shared library: [libmpc.so.2]
  # ls -l /var/tmp/catalyst/tmp/default/stage1-i686-2013.1/tmp/stage1root/usr/libexec/gcc/i686-pc-linux-gnu/4.6.3/cc1 
  -rwxr-xr-x 1 root root 10557140 Apr 11 21:24 /var/tmp/catalyst/tmp/default/stage1-i686-2013.1/tmp/stage1root/usr/libexec/gcc/i686-pc-linux-gnu/4.6.3/cc1

Yup.  And that mpc version isn't in the new stage1:

  # ls /var/tmp/catalyst/tmp/default/stage1-i686-2013.1/tmp/stage1root/usr/lib/libmpc.so*
  /var/tmp/catalyst/tmp/default/stage1-i686-2013.1/tmp/stage1root/usr/lib/libmpc.so
  /var/tmp/catalyst/tmp/default/stage1-i686-2013.1/tmp/stage1root/usr/lib/libmpc.so.3
  /var/tmp/catalyst/tmp/default/stage1-i686-2013.1/tmp/stage1root/usr/lib/libmpc.so.3.0.0

So, while iamben (I think) and I (definitely) ran into this problem
because we had just turned on update_seed (with out-of-date packeages
from a non-update_seed run), this problem can also crop up due to
snapshot updates (even if you always have update_seed on).  That means
that the only reasonable fixes are:

a. Clear the package cache whenever you update a spec (without bumping
   version_stamp).
b. Migrate the toolchain packages to EAPI5 and use ABI sub-slotting.

Since (b), even if it does happen, is likely to take a while, I think
we should be pushing (a) for now.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy