| 1 |
My experience with NFS mounted roots is that they can bombard your |
| 2 |
network with packets. A simple little script can manage to generate |
| 3 |
enough traffic to actually slow down other services just by hitting |
| 4 |
tons of services. Plus if you have spoolers and such you end up |
| 5 |
generating a ton of traffic or memory. The advantage to a local |
| 6 |
install is that you can cut down on the network traffic drastically. |
| 7 |
Granted, if your applications are all embarrassingly parallel and |
| 8 |
don't do a ton of disk IO then NFS root works great.. Many of the |
| 9 |
applications we use here would utterly destroy the network if run |
| 10 |
from a NFS mounted root.. The advantage of rebuilding is consistency |
| 11 |
without the disadvantage of NFS roots. |
| 12 |
|
| 13 |
|
| 14 |
On Nov 21, 2005, at 9:45 AM, Eric Thibodeau wrote: |
| 15 |
|
| 16 |
> Le 21 Novembre 2005 11:10, Robin H. Johnson a écrit : |
| 17 |
>> On Sun, Nov 20, 2005 at 08:51:13PM -0500, St?phane Lacasse wrote: |
| 18 |
>> [snip discussion about installing] |
| 19 |
>> |
| 20 |
>> I've done the cluster system (128 node+ 1 master) in a similar |
| 21 |
>> fashion |
| 22 |
>> to what you are after. |
| 23 |
>> 1. PXE-boot install environment for performing installs of both the |
| 24 |
>> master and all of the nodes. |
| 25 |
> PXE-boot even for the Master?...so where do the images reside...how |
| 26 |
> do you |
| 27 |
> manage the slightly varying config items such as hostname and all? |
| 28 |
> This |
| 29 |
> approach still seems a little bit time consuming since all nodes |
| 30 |
> are still |
| 31 |
> individual entities (not NFS roots to a single maintained image). |
| 32 |
> Though |
| 33 |
> granted that the nodes being all identical, emerge -K should in |
| 34 |
> theory be a |
| 35 |
> breeze....but it's not the case for maintaining all the config files |
| 36 |
> consistent. |
| 37 |
> |
| 38 |
>> 2. The install environment uses the Gentoo Installer, with the CLI |
| 39 |
>> frontend I wrote for the GLI project, and performs complete |
| 40 |
>> installs of |
| 41 |
>> nodes in under 20 minutes (depending on network traffic). |
| 42 |
> So switching a machine's purpose/profile requires a complete re- |
| 43 |
> install on the |
| 44 |
> node? You state 20 minutes for re-installing, is it a _real_ |
| 45 |
> install or the |
| 46 |
> dump of a "reference" root? (Pardon my ignorance of the CLI |
| 47 |
> installer you are |
| 48 |
> referring to... I'll read the http link you'll send me ;) ) |
| 49 |
> |
| 50 |
>> By using GLI, it's a simple matter of altering the install |
| 51 |
>> profiles to |
| 52 |
>> reconfigure the cluster, and wipe the nodes for changing their |
| 53 |
>> purpose |
| 54 |
>> (presently we have an MPI mode and a MOSIX mode), some of the cluster |
| 55 |
>> users need assurances that none of their data remains on the cluster |
| 56 |
>> after they are done, hence being able to reinstall easily. |
| 57 |
> [...] |
| 58 |
>> Also, make use of your cluster tools to administer the cluster. |
| 59 |
>> OpenPBS |
| 60 |
>> allows running a job on all nodes, so use it to emerge -K [package]. |
| 61 |
>> (not -k as binpkgs don't currently have any locking in $PKGDIR, |
| 62 |
>> and can |
| 63 |
>> get corrupted if two emerge processes try to create a binpkg at the |
| 64 |
>> same time.) |
| 65 |
> |
| 66 |
> Actually, I would have thought you use _one_ node to compile the |
| 67 |
> packages |
| 68 |
> (using distcc at your description) and _then_ propagate the package |
| 69 |
> onto the |
| 70 |
> other nodes with -K....still, I would think maintaining an NFS |
| 71 |
> mounted ROOT |
| 72 |
> would be less cumbersome.... |
| 73 |
> |
| 74 |
> -- |
| 75 |
> Eric Thibodeau |
| 76 |
> |
| 77 |
> -- |
| 78 |
> gentoo-cluster@g.o mailing list |
| 79 |
> |
| 80 |
|
| 81 |
|
| 82 |
-- |
| 83 |
gentoo-cluster@g.o mailing list |
Archives