Re: [gentoo-soc] Re: GSoc 2009 : Making the cluster LiveCD bootable from USB

[Eric Thibodeau]

Tom Stellard wrote:
> On Mon, Mar 23, 2009 at 7:18 AM, Eric Thibodeau <kyron@××××××××.com> wrote:
>
>> In response to two e-mails I just got off list from Meetanshu Gupta  and
>> Ajith Kamath, in both cases the impression is that the project is simply to
>> port the bootable CD to USB disk. Here is are some clarifications:
>>
>> - The wiki's¹ description is misleading, booting to USB was ONE of my
>> suggestions with a little more details here². Quite frankly, porting to USB
>> should be somewhat trivial. The hardest part is to being very methodical and
>> making sure the process can be reproduced using, ideally, Catalyst and
>> little outside scripting. All of this also implies getting the catalyst
>> specs up to date, modifying/removing many hacks that would no longer be
>> required for a USB based boot (since we would now have an RW system). ***
>> root on JFFS2 might be of interest here ***... and so on.
>>
>>
>
> Is the idea that booting from a USB device would be added as an option
> when building the liveCD image, or is booting from a USB device
> supposed to replace booting from a CD?
>
The idea is to replace the CD. CDs are a PITA to use from an OS point of 
view and require overlayed over overlayed filesystems. Given the size 
price and availability of USB keys these days, it's just no longer worth 
the effort. Using a USB-based install also makes the installation 
"upgradable" and an excellent starting point that can evolve. We should 
then consider having the portage tree included in the image or make it 
easy to add it (ie: don't strip too much out of the image or have 
alternate images...which is a nuisance IMHO).
>
>> - AuFS is used as an overlay system and is required for the NFS-booted
>> nodes. One might want to migrate that to a less efficient tmpfs approach
>> (ref: LTSP implementation as being done under Gentoo too).
>> - Change most of the scripts to use dialog and automate some detection/ask
>> questions as per what was discovered.
>> - ... and so on, the idea would be to implement my original proposal and
>> push it further. I do have it but want some feedback before I dig it out and
>> re-post it (Donnie, thought you had posted it as an example? If not so, feel
>> free to do so if it's still relevant.)
>>
>
> I have been working on building the liveCD the last few days, and I am
> almost finished.  I have run into many problems along the way, and I
> have come up with a few general ideas for things to include in my
> proposal:
>
> -Add spec files for x86.
>
yes
> -Update the hpc-clustering and nfsroot profiles and the catalyst spec files.
>
yes
> -Simplify and automate the build process with a configuration script.
>
? Meaning that you'd make a script that calls catalyst? Be my guest ;)
> -Add documentation for building the liveCD using the new configuration script.
>
Or liveMedium as I started calling it for USB usage (or external HDD ;). 
Documentation is _always_ a good thing.
> These tasks all involve the build process, and once they have been
> completed, work could begin on USB booting, JFFS2 or other areas of
> interest.
>
Booting off USB should be the priority, even before nice docs or 
automating the calling of catalyst. I didn't make a script for building 
the images since the script does very little once the environment is set up.
> It seems like a lot of work has been done to make sure the right
> packages are included in the build, and that currently the biggest
> challenge is updating the build process and making it easy to
> duplicate.
I don't know the status of the new version of Catalyst but many new 
features, such as building stages 1-3 or even 1-4 in a single call, are 
supposed to be added. I suggest checking out the status of that project 
and then planning the "move" if applicable. Otherwise we'll have to 
stick to the hackerish scripts and catalyst specs.

My notes on why duplicating the build process tend to point to the 
portage tree being so dynamic and the stage builds being very sensitive 
to the tree. With the advent of snapshots and stages being produced on a 
regular basis, the stage build pains should be lessened significantly 
compared to the quarterly or bi-annual releases.
>  Will improving the build process, USB booting and the
> related file system changes be enough work for one summer, or are
> there other parts of the cluster liveCD that Gentoo developers would
> like to see improved?
>
Most of the configuration script(s) should be modified into a dialog(1) 
based script where the user is presented options that would be selected. 
The setup script I made is quite crude and reads in 2 config files. This 
should be changed to use dialog and provide the user the opportunity to 
change the default values.

Another significant leap would be to be able to boot the nodes off 
NFSv4. This may sound trivial but the kernel deosn't support this 
natively which implies using a ramdisk (initrd). NFSv4 has many advantages:

    * Ability to use links outside of the exported root (server side):
    -- this means that most of the common config files between the 
server and nodes could be links...very _interesting_
    * better caching
    * all sorts of new goodies I am forgetting about

The problem with this might be that the overlayed filesystem used for 
the nfsroot might not like NFSv4 (aufs is already having a hard time 
with NFS at all). This might be yet another reason to use tmpfs+bind 
mounts*¹* as being done for the Gentoo MueCow project.

> -Tom Stellard
>
So, there is still much to be done OS side even before we start looking 
at goodies such as integrating TAU² and other popular scientific 
apps...which mostly also imply good X drivers with all the surrounding 
issues... We might even have enough meat for more than one SoCer on this :P.

Eric Thibodeau

¹ http://www.funtoo.org/en/articles/linux/ffg/3/ "Tmpfs and bind mounts"
² http://www.cs.uoregon.edu/research/tau/home.php