1 |
You mentioned DrQueue, that batch processing app fulfills many of your |
2 |
requirements. I wrote an ebuild for it too which is on bugs.gentoo |
3 |
|
4 |
Hanni |
5 |
|
6 |
On 11/01/2008, Jos Houtman <jos@×××××.nl> wrote: |
7 |
> |
8 |
> Thanx for the replies, |
9 |
> |
10 |
> I will certainly try the beowulf mailinglist next week, |
11 |
> The suggestions are great gearman seems to fit except for being perl |
12 |
> oriented (atleast that's what I read sofar). |
13 |
> Torque/pbs, don't know about that yet. |
14 |
> |
15 |
> Thanx again, |
16 |
> |
17 |
> Jos Houtman |
18 |
> System administrator Hyves.nl |
19 |
> email: jos@×××××.nl |
20 |
> |
21 |
> -----Original Message----- |
22 |
> From: Robin H. Johnson [mailto:robbat2@g.o] |
23 |
> Sent: donderdag 10 januari 2008 15:30 |
24 |
> To: gentoo-cluster@l.g.o |
25 |
> Subject: Re: [gentoo-cluster] cluster or distributed queue, general |
26 |
> question |
27 |
> |
28 |
> On Thu, Jan 10, 2008 at 02:59:27PM +0100, Jos Houtman wrote: |
29 |
> > For my master thesis I took up a project that requires mapping of a |
30 |
> number of statically defined parallel jobs into a more dynamic |
31 |
> environment that allows better scaling. |
32 |
> > The situation as described below let me to believe a cluster or |
33 |
> distributed queue (DrQueue?) solution is necessary. For the situation |
34 |
> see [situation] at the end of this email. |
35 |
> Off the top of my head, many of your requirements are available in two |
36 |
> totally different apps: |
37 |
> - Gearman, written by Brad Fitzpatrick @ LiveJournal. Perl mainly, I |
38 |
> think there are other interfaces as well to it. |
39 |
> - Torque/PBS - somewhat less of a fit, I'm not certain about running |
40 |
> perpetual jobs. |
41 |
> |
42 |
> You may also need some degree of STONITH for the job running only once |
43 |
> during node failure case. (Say the job manager crashes, the job is still |
44 |
> running, but you have no control of it. You need to zap it hard). |
45 |
> |
46 |
> -- |
47 |
> Robin Hugh Johnson |
48 |
> Gentoo Linux Developer & Infra Guy |
49 |
> E-Mail : robbat2@g.o |
50 |
> GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 |
51 |
> -- |
52 |
> gentoo-cluster@l.g.o mailing list |
53 |
> |
54 |
> |
55 |
|
56 |
|
57 |
-- |
58 |
E-mail: hanni.ali@×××××.com |
59 |
Mobile: +44 (0) 7985580147 |
60 |
My Blog: http://ainkaboot.co.uk/blogs/hanni/ |
61 |
Website: http://ainkaboot.co.uk http://drqueue.org |