| 1 |
-----BEGIN PGP SIGNED MESSAGE----- |
| 2 |
Hash: SHA256 |
| 3 |
|
| 4 |
On 12/09/12 05:55 AM, Gregory M. Turner wrote: |
| 5 |
> |
| 6 |
> Note that, effectively, we have this already, and it's called |
| 7 |
> "portage". But one could certainly make a case for modularizing it |
| 8 |
> better, since, in truth, we are talking about a very common, very |
| 9 |
> abstract problem here which portage shares with any number of |
| 10 |
> batch-build systems. |
| 11 |
> |
| 12 |
> Such an engine could very well do exactly the right thing if it |
| 13 |
> were faced with a constraint that a certain part of a certain build |
| 14 |
> needed to proceed without parallelism due to limitations coming |
| 15 |
> from the build. |
| 16 |
> |
| 17 |
> Also, there are very large parts of most builds -- configure comes |
| 18 |
> to mind -- that don't parallelize even if, perhaps, they should. |
| 19 |
> In such cases, a really smart global parallelism arbiter could |
| 20 |
> easily respond by spawning more jobs from other builds. |
| 21 |
> |
| 22 |
|
| 23 |
So essentially what you're saying here is that it might be worthwhile |
| 24 |
to look into parallelism as a whole and possibly come up with a |
| 25 |
solution that combines 'emerge --jobs' and build-system parallelism |
| 26 |
together to maximum benefit? |
| 27 |
|
| 28 |
Advanced HPC systems (sys-cluster/torque along with an appropriate |
| 29 |
scheduler, for instance) can do such things with their jobs when the |
| 30 |
jobs are properly built; I could see portage being able to handle this |
| 31 |
as well given most of what is necessary is already known (ebuild |
| 32 |
phases, build system type (via eclass), etc). However, given the |
| 33 |
limitations already put on parallelism in terms of emerge order, etc, |
| 34 |
I could see this solution needing to be -very- complex and integration |
| 35 |
needing to occur on multiple levels. We'd also need to consider |
| 36 |
distcc (and other cluster-shared compilation methods if there are |
| 37 |
any??).. It would be an interesting project, though. |
| 38 |
-----BEGIN PGP SIGNATURE----- |
| 39 |
Version: GnuPG v2.0.19 (GNU/Linux) |
| 40 |
|
| 41 |
iF4EAREIAAYFAlBQhwAACgkQ2ugaI38ACPCKrgD+JNlHPUl7ETYDC6u3lYWRSz8J |
| 42 |
fpWC/puDfCYu51yNOVIA/0E+U6x9Ds8GV8r/RinkTqss3/fcd06w24GRvZOda3Mj |
| 43 |
=XONZ |
| 44 |
-----END PGP SIGNATURE----- |
Archives