[gentoo-cluster] examples of (large) Gentoo clusters

[John R. Dunning]

From: "Daniel van Ham Colchete" <daniel.colchete@×××××.com>
    Date: Mon, 4 Dec 2006 21:55:12 -0200
[...]    
	    Lustre uses a lot of kernel features that if not enabled will
    cause the kernel to crash.
[...]    
I don't think that's true.

I've been running lustre on assorted kernels, mostly under gentoo dists, for
some months, and found that once you get past the installation issues, it's
pretty trouble free.

Now, note the caveats there: The installation issues are non-trivial, mostly
because lustre is very intrusive into the vfs layer.  This causes no end of
headaches integrating with various other peoples' kernel patches, due to
collisions with the other peoples' patches to vfs.  That statement is as true
of gentoo as it is of any kernel other than the few for which they supply
canned patchsets.  But I've never seen anything in there that contitutes using
a kernel feature which causes the kernel to crash if not enabled.  The closest
thing I've seen to that is if you muff the patch merging and end up with an
inconsistent patchset, that generally leads to a crash :-}

Lustre 1.6 (at least the client end) doesn't even really *require* all those
kernel patches, ie they do support the idea of a patchless client.  The issue
is that lustre changes the logic involved in various kinds of fs operations,
including anything related to lookups, so as to short-circuit much of the work
involved when it figures out that it can do so.  Running the client without
the patches will work, but it won't give you the performance that you'd get
with the patches.  So odds are anybody who's interested in running lustre in
the first place probably wants the patches too.

Lustre also is not restricted to precompiled kernels, their build script
contains patchsets for things other than their recommended redhat and suse
ones.  We routinely compile it up for all kinds of experimental kernels with
no trouble.  Again, once you've gotten over the hurdle of getting the kernel
patches integrated, the rest of it behaves reasonably well.  The reason cfs
advocates the small number of kernels they do is because they know what a pain
it is to draw outside the lines, and they try to steer people away from that.

We at sicortex are planning on rolling out a gentoo-based cluster that depends
heavily on lustre, so we've spent a fair bit of time banging on it.  I'm
pretty sure we understand it at this point.  We'll know for sure soon :-}
-- 
gentoo-cluster@g.o mailing list