| 1 |
On Thu, Jan 10, 2008 at 02:59:27PM +0100, Jos Houtman wrote: |
| 2 |
> For my master thesis I took up a project that requires mapping of a number of statically defined parallel jobs into a more dynamic environment that allows better scaling. |
| 3 |
> The situation as described below let me to believe a cluster or distributed queue (DrQueue?) solution is necessary. For the situation see [situation] at the end of this email. |
| 4 |
Off the top of my head, many of your requirements are available in two |
| 5 |
totally different apps: |
| 6 |
- Gearman, written by Brad Fitzpatrick @ LiveJournal. Perl mainly, I |
| 7 |
think there are other interfaces as well to it. |
| 8 |
- Torque/PBS - somewhat less of a fit, I'm not certain about running |
| 9 |
perpetual jobs. |
| 10 |
|
| 11 |
You may also need some degree of STONITH for the job running only once |
| 12 |
during node failure case. (Say the job manager crashes, the job is still |
| 13 |
running, but you have no control of it. You need to zap it hard). |
| 14 |
|
| 15 |
-- |
| 16 |
Robin Hugh Johnson |
| 17 |
Gentoo Linux Developer & Infra Guy |
| 18 |
E-Mail : robbat2@g.o |
| 19 |
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 |
Archives