Gentoo Archives: gentoo-dev

From: Patrick McLean <chutzpah@g.o>
To: gentoo-dev@l.g.o
Cc: TomWij@g.o, patrick@g.o
Subject: Re: [gentoo-dev] Portage QOS
Date: Fri, 10 Jan 2014 01:52:28
Message-Id: 20140109175216.5c558257@gentoo.org
In Reply to: Re: [gentoo-dev] Portage QOS by Tom Wijsman
1 On Fri, 10 Jan 2014 02:19:03 +0100
2 Tom Wijsman <TomWij@g.o> wrote:
3
4 > On Fri, 10 Jan 2014 08:31:21 +0800
5 > Patrick Lauer <patrick@g.o> wrote:
6 >
7 > > On 01/10/2014 08:16 AM, heroxbd@g.o wrote:
8 > > > Igor <lanthruster@×××××.com> writes:
9 > > >
10 > > >> The ebuilds have approximately the same time to install, the
11 > > >> failure rate is about the same, emerge is getting slower.
12 > > >
13 > > > I am curious about the slowness of emerge.
14 > > >
15 > > > How about profile the portage and rewrite the time-crucial part in
16 > > > C/C++, or ideally, borrowing the counterpart from paludis? How
17 > > > feasible is that?
18 > >
19 > > Last I checked paludis wasn't faster - on average portage was a few
20 > > percents faster.
21 >
22
23 > > For python things you really want python or C instead of C++...
24 >
25 > Why is this a Python thing? What's the reason to exclude a language?
26 >
27 > > So, what you wanted to ask was:
28 > > "Which parts of pkgcore can be migrated into portage?"
29 >
30 > Or rather: "What does it take to migrate parts of pkgcore into
31 > portage?"
32
33 Why not just switch to using pkgcore as the default package manager.
34 radhermit has been doing a lot of work lately getting EAPI 5 support
35 added, and generally fixing bugs etc.
36
37 Since we no longer have anyone intimately familiar with the
38 portage code, we should just switch to a more modern and readable
39 system. Rather than having random people trying to learn the convoluted
40 portage code, let's concentrate on getting pkgcore to a point where
41 we can replace portage with it.
42
43 > > > I guess the dep-tree calculation is the slowest part.
44 > > Yes, it's doing lots of silly dynamic things (backtracking),
45 >
46 > Exactly, without backtracking (which I can live without) it takes
47 > around half a minute as compared to a lot of minutes; when it started
48 > to take almost half an hour it was time to completely nuke
49 > backtracking.
50 >
51 > Now I happily live without ever needing to enable backtracking
52 > again. :)
53 >
54 > > and portage codebase on average is not designed for speed.
55 >
56 > Yes, inspecting the profiler results has me worried; though I find
57 > half a minute acceptable for the amount of packages that are involved
58 > on my system, so, I'm less concerned but it is still interesting that
59 > there is the possibility to do things faster. It's just some work to
60 > actually get it to do that, which requires much more dedication, time
61 > and knowledge than doing an occasional commit.
62 >
63 > > As a first step I would recommend profiling it and removing unneeded
64 > > stuff (do less work!), rewriting parts in C is a lot of work and not
65 > > needed for the first round of speedups.
66 >
67 > See my other mail <20140110020218.0f6244d5@TOMWIJ-GENTOO> for recent
68 > profiling images; we should indeed start with dealing with the low
69 > hanging fruit (there are quite some 0-2% speed improvements possible),
70 > as that can get us to speed it up faster than trying to deal with some
71 > algorithmic complexity that needs far more insight to redesign, let
72 > alone to properly re-implement it.
73 >

Replies

Subject Author
Re: [gentoo-dev] Portage QOS Tom Wijsman <TomWij@g.o>
Re: [gentoo-dev] Portage QOS Brian Dolbec <dolsen@g.o>
Re: [gentoo-dev] Portage QOS Ciaran McCreesh <ciaran.mccreesh@××××××××××.com>