| 1 |
On Thu, Apr 25, 2013 at 11:58 AM, áÌÅËÓÁÎÄÒ âÅÒÓÅÎÅ× <bay@×××××××××.ru> wrote: |
| 2 |
> Hello, |
| 3 |
> |
| 4 |
> my name is Alexander Bersenev, I am postgraduate of Institute of Mathematics |
| 5 |
> and Mechanics(Russia). |
| 6 |
|
| 7 |
Hello, it's nice to meet you. |
| 8 |
|
| 9 |
> I want to propose a project for GSoC 2013 and ask what do you think about |
| 10 |
> it. |
| 11 |
> |
| 12 |
> In short: I want to reduce the "Calculating dependencies" phase of emerge. |
| 13 |
> |
| 14 |
> On my notebook "emerge -pv bash" command takes 40 secs to calculate a deps. |
| 15 |
> If I launch it again, it take about 40 secs again(a have a lot of RAM, so |
| 16 |
> there was no HDD usage). |
| 17 |
|
| 18 |
A few things to note: |
| 19 |
|
| 20 |
1) It will make a big difference if there is a bash version upgrade, |
| 21 |
or if the bash USE flags have changed. This is due to the |
| 22 |
--complete-graph-if-new-use and --complete-graph-if-new-ver options |
| 23 |
which are enabled by default. This behavior serves to protect |
| 24 |
reverse-dependencies from being broken. |
| 25 |
|
| 26 |
2) Portage assumes that the portage tree can be modified between each |
| 27 |
emerge invocation. This is assumption necessary for development |
| 28 |
situations, but it has the disadvantage of introducing some extra |
| 29 |
overhead (comparing checksums of ebuilds and eclasses to the checksums |
| 30 |
found in the corresponding md5-cache entries). It would be possible to |
| 31 |
have an alternative "frozen tree" mode of operation which assumes that |
| 32 |
the portage tree can _not_ be modified between emerge invocations, and |
| 33 |
this mode would be more optimal for non-development situations. |
| 34 |
|
| 35 |
3) Putting the portage tree on squashfs can help in some situations, |
| 36 |
since it allows the whole tree to easily fit into RAM and be accessed |
| 37 |
quickly. |
| 38 |
|
| 39 |
> Of course, quick cprofile profiling showed no places to optimize because |
| 40 |
> such optimizations already have been made. |
| 41 |
> |
| 42 |
> The main idea is add some caching layers(more high-level, than in |
| 43 |
> /usr/portage/metadata/md5-cache/). The main goal is to find and eliminate |
| 44 |
> repeated computations between "emerge" runs. |
| 45 |
> |
| 46 |
> As part of work I plan to examine approaches of other pkg managers(yum, |
| 47 |
> aptitude). |
| 48 |
> |
| 49 |
> I heard from Donnie Berkholz in IRC about pkgcore project. He said it works |
| 50 |
> faster in practice. But it has some problems with EAPI5 support. |
| 51 |
> |
| 52 |
> What is better: actualize a pkgcore code or try to dig into portage? Or it |
| 53 |
> is |
| 54 |
> the bad ideas at all? |
| 55 |
|
| 56 |
I suspect the pkgcore may already have a "frozen tree" mode, among |
| 57 |
other optimizations. However, it's not very useful until EAPI 5 |
| 58 |
support is completed. |
| 59 |
|
| 60 |
Adding "frozen tree" support to portage might be a nice enhancement, |
| 61 |
but I'm not sure how much performance increase that it would yield. |
| 62 |
The --complete-graph-* options that I've mentioned introduce a large |
| 63 |
amount of overhead that could easily overshadow any performance |
| 64 |
increase that a "frozen tree" optimization would give you. |
Archives