| 1 |
Ned Ludd wrote: |
| 2 |
> With the size of the vdb effecting the speed of portage itself and with |
| 3 |
> all the desktop splitting up packages into massively large sets of many |
| 4 |
> ebuilds it's becoming apparent that that some optimizations need to be |
| 5 |
> done. |
| 6 |
> |
| 7 |
> strace -o woof -eopen -f python -c 'import portage' |
| 8 |
> grep '/var/db/pkg/sys-apps/portage-' woof |
| 9 |
> |
| 10 |
> You will notice that on a simple import that it's reading everything |
| 11 |
> twice. If anybody is up for the challenge of addressing this double |
| 12 |
> reading I think the entire user community would be thankful |
| 13 |
> (or atleast I would be) |
| 14 |
> |
| 15 |
> The time it takes to import portage with only 78 pkgs installed takes |
| 16 |
> about 1 second. On the same box in another chroot it takes over 4-5 |
| 17 |
> seconds with about 400 packages installed. I hear of some guys that have |
| 18 |
> over 1000 packages installed. That must be a nightmare especially with |
| 19 |
> packages that make heavy use of the portageq command. |
| 20 |
> |
| 21 |
|
| 22 |
Well, we all know that the second read is quicker than the first due to kernel caching. ;) |
| 23 |
|
| 24 |
But anyway, out of curiousity, I did the following: |
| 25 |
|
| 26 |
sandboxshell |
| 27 |
adddeny /var/db/pkg |
| 28 |
python -c 'import portage' |
| 29 |
|
| 30 |
>From the resulting traceback(s), I was able to see that the /var/db/pkg reads originate in calls to a getvirtuals() method, which ultimately leads to a get_all_provides() method. Assuming that the results from get_all_provides() do not change during the course of the initial 'import portage', it should be safe to cache the results (until the end of the import). |
| 31 |
|
| 32 |
Zac |
| 33 |
-- |
| 34 |
gentoo-portage-dev@g.o mailing list |
Archives