Gentoo Archives: gentoo-dev

From: Abraham Marin Perez <abraham@×××××××××.es>
To: gentoo-dev@l.g.o
Subject: [gentoo-dev] Proposal of change in emerge
Date: Sun, 26 Sep 2004 14:06:11
Message-Id: 1096207905.27639.140.camel@localhost
1 Hi everyone,
2
3 I've been thinking about a new way to make emerge works, it's just
4 an idea and I haven't thought about the code itself, and actually I have
5 never used python nor hacked portage code, but I thought it'd be good to
6 tell it. Here it goes:
7
8 Let's say we want to install a package X and that it needs packages
9 Y and Z to be emerged. However, Y and Z don't depend in each other, so
10 we can do some kind of parallel merging to speed up the whole process:
11 we download Y and then we start Y's compilation, but, since we don't use
12 the network during the compilation, we can safely download Z meanwhile.
13 When Y's compilation and Z's downloading are over, we can compile Z and
14 download X while Y is being merged.
15
16 This example is quite simple, but that's the idea: find out which
17 packages are independent and which should wait for some others to be
18 merged, separate the whole process in parts and run in parallel
19 independent tasks. Portage has now a main core who calculates
20 dependences and "independences" and uses three agents: the downloader,
21 the builder and the installer. If we have something to download we check
22 whether the downloader is free, if so, we send it the task. If the
23 compiler is free and we have an already downloaded file that doesn't
24 need any other package to be merged, we send it the task. If the
25 installer is free and we have a compiled packaged, we use the installer
26 to merge it.
27
28 On the other hand, if you try to emerge more than one package at the
29 same time and they happen to share dependences they may try to emerge
30 them simultaneously, causing several problems. Hence, we could make
31 portage works like a server that merges simultaneous install petitions
32 and check for shared dependences. Of course, it should keep the original
33 package to know what should be added to world file and which tasks
34 should be stopped if the main job is cancelled.
35
36 I hope I didn't state the obvious, as I said, I haven't taken a look
37 to portage code but, according the way it works, this new way could make
38 it faster and avoid some problems. What do you think about it?
39
40 Regards,
41 Abraham
42
43
44 ************************************************************************
45 Abraham Marin Perez
46 Studying Computer Science at the University of Valencia
47 E-mail: abraham@×××××××××.es
48 Home page: http://mural.uv.es/abraham
49
50 "Sorry, this virus requires Microsoft Windows", Blaster.C
51 ************************************************************************
52
53
54 --
55 gentoo-dev@g.o mailing list

Replies

Subject Author
Re: [gentoo-dev] Proposal of change in emerge Ciaran McCreesh <ciaranm@g.o>