Re: [gentoo-portage-dev] portage perfoemance again ...

[Zac Medico]

On 03/06/2018 02:10 PM, Joakim Tjernlund wrote:
> .. using portage 2.3.24 on our embedded target(ppc329 It takes forever, > 10 mins) to just do the
> dependencies for emerge -aNDuv world
> Here is what strace says, one can see portage is rereading the profile over and over again.
> At this point I am prepared to try anything to eliminate the rereading of the profile, any pointers?
>
> 23:04:25.472818 _llseek(5, 0, [0], SEEK_CUR) = 0
> 23:04:25.472978 read(5, "# Copyright 1999-2015 Gentoo Fou"..., 32768) = 677
> 23:04:25.473099 read(5, "", 28672)      = 0
> 23:04:25.473228 read(5, "", 32768)      = 0
> 23:04:25.473366 close(5)                = 0
> 23:04:25.482102 stat64("/usr/portage/profiles/base/profile.bashrc", {st_mode=S_IFREG|0644, st_size=183, ...}) = 0
> 23:04:25.482509 stat64("/usr/portage/profiles/default/linux/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
> 23:04:25.482887 stat64("/usr/portage/profiles/arch/base/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
> 23:04:25.483240 stat64("/usr/portage/profiles/arch/powerpc/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
> 23:04:25.483594 stat64("/usr/portage/profiles/default/linux/powerpc/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
> 23:04:25.483924 stat64("/usr/portage/profiles/arch/base/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
> 23:04:25.484269 stat64("/usr/portage/profiles/arch/powerpc/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
> 23:04:25.484604 stat64("/usr/portage/profiles/arch/powerpc/ppc32/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
> 23:04:25.484930 stat64("/usr/portage/profiles/default/linux/powerpc/ppc32/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
> 23:04:25.485272 stat64("/usr/portage/profiles/releases/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
> 23:04:25.485615 stat64("/usr/portage/profiles/releases/13.0/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
> 23:04:25.485944 stat64("/usr/portage/profiles/default/linux/powerpc/ppc32/13.0/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)
> 23:04:25.486329 stat64("/usr/local/portage/tmv3-target-overlay/profiles/cusfpv3/profile.bashrc", {st_mode=S_IFREG|0644, st_size=133, ...}) = 0
> 23:04:25.486764 stat64("/etc/portage/profile/profile.bashrc", 0xbfe61330) = -1 ENOENT (No such file or directory)

The profile.bashrc stat calls are repeated for every setcpv call here:

https://gitweb.gentoo.org/proj/portage.git/tree/pym/portage/package/ebuild/config.py?h=portage-2.3.24#n1614

We can easily perform the stat calls in the constructor, and save the
results in an attribute, like when we initialize self._pbashrcdict.

Bug filed: https://bugs.gentoo.org/649806
-- 
Thanks,
Zac