Gentoo Archives: gentoo-portage-dev

From: Joakim Tjernlund <Joakim.Tjernlund@××××××××.com>
To: "zmedico@g.o" <zmedico@g.o>, "gentoo-portage-dev@l.g.o" <gentoo-portage-dev@l.g.o>
Subject: Re: [gentoo-portage-dev] portage perfoemance again ...
Date: Wed, 07 Mar 2018 00:12:01
Message-Id: 1520381515.4790.42.camel@infinera.com
In Reply to: Re: [gentoo-portage-dev] portage perfoemance again ... by Zac Medico
1 On Tue, 2018-03-06 at 15:11 -0800, Zac Medico wrote:
2 > On 03/06/2018 02:10 PM, Joakim Tjernlund wrote:
3 > > .. using portage 2.3.24 on our embedded target(ppc329 It takes forever, > 10 mins) to just do the
4 > > dependencies for emerge -aNDuv world
5 > > Here is what strace says, one can see portage is rereading the profile over and over again.
6 > > At this point I am prepared to try anything to eliminate the rereading of the profile, any pointers?
7 > >
8 > > 23:04:25.472818 _llseek(5, 0, [0], SEEK_CUR) = 0
9 > > 23:04:25.472978 read(5, "# Copyright 1999-2015 Gentoo Fou"..., 32768) = 677
10 > > 23:04:25.473099 read(5, "", 28672) = 0
11 > > 23:04:25.473228 read(5, "", 32768) = 0
12 > > 23:04:25.473366 close(5) = 0
13 > > 23:04:25.482102 stat64("/usr/portage/profiles/base/profile.bashrc", {st_mode=S_IFREG|0644, st_size=183, ...}) = 0
14 > > 23:04:25.482509 stat64("/usr/portage/profiles/default/linux/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
15 > > 23:04:25.482887 stat64("/usr/portage/profiles/arch/base/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
16 > > 23:04:25.483240 stat64("/usr/portage/profiles/arch/powerpc/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
17 > > 23:04:25.483594 stat64("/usr/portage/profiles/default/linux/powerpc/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
18 > > 23:04:25.483924 stat64("/usr/portage/profiles/arch/base/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
19 > > 23:04:25.484269 stat64("/usr/portage/profiles/arch/powerpc/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
20 > > 23:04:25.484604 stat64("/usr/portage/profiles/arch/powerpc/ppc32/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
21 > > 23:04:25.484930 stat64("/usr/portage/profiles/default/linux/powerpc/ppc32/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
22 > > 23:04:25.485272 stat64("/usr/portage/profiles/releases/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
23 > > 23:04:25.485615 stat64("/usr/portage/profiles/releases/13.0/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
24 > > 23:04:25.485944 stat64("/usr/portage/profiles/default/linux/powerpc/ppc32/13.0/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
25 > > 23:04:25.486329 stat64("/usr/local/portage/tmv3-target-overlay/profiles/cusfpv3/profile.bashrc", {st_mode=S_IFREG|0644, st_size=133, ...}) = 0
26 > > 23:04:25.486764 stat64("/etc/portage/profile/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
27 >
28 > The profile.bashrc stat calls are repeated for every setcpv call here:
29 >
30 > https://gitweb.gentoo.org/proj/portage.git/tree/pym/portage/package/ebuild/config.py?h=portage-2.3.24#n1614
31 >
32 > We can easily perform the stat calls in the constructor, and save the
33 > results in an attribute, like when we initialize self._pbashrcdict.
34 >
35 > Bug filed: https://bugs.gentoo.org/649806
36
37 Nice! One you got a patch I will try it out.
38
39 Desperate, I downgraded to portage 2.2.26 and now portage runs a lot faster! No measurements
40 though as I need to complete the upgrade of our system ASAP.
41 Does this make any sense to you?
42
43 Jocke