<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Tom Stellard wrote:
<blockquote
cite="mid:d24863920903222250g7f755039m9cefe643d48cc1f2@..."
type="cite">
<pre wrap="">On Mon, Mar 23, 2009 at 7:18 AM, Eric Thibodeau <a class="moz-txt-link-rfc2396E" href="mailto:kyron@..."><kyron@...></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">In response to two e-mails I just got off list from Meetanshu Gupta and
Ajith Kamath, in both cases the impression is that the project is simply to
port the bootable CD to USB disk. Here is are some clarifications:
- The wiki's¹ description is misleading, booting to USB was ONE of my
suggestions with a little more details here². Quite frankly, porting to USB
should be somewhat trivial. The hardest part is to being very methodical and
making sure the process can be reproduced using, ideally, Catalyst and
little outside scripting. All of this also implies getting the catalyst
specs up to date, modifying/removing many hacks that would no longer be
required for a USB based boot (since we would now have an RW system). ***
root on JFFS2 might be of interest here ***... and so on.
</pre>
</blockquote>
<pre wrap=""><!---->
Is the idea that booting from a USB device would be added as an option
when building the liveCD image, or is booting from a USB device
supposed to replace booting from a CD?
</pre>
</blockquote>
The idea is to replace the CD. CDs are a PITA to use from an OS point
of view and require overlayed over overlayed filesystems. Given the
size price and availability of USB keys these days, it's just no longer
worth the effort. Using a USB-based install also makes the installation
"upgradable" and an excellent starting point that can evolve. We should
then consider having the portage tree included in the image or make it
easy to add it (ie: don't strip too much out of the image or have
alternate images...which is a nuisance IMHO).<br>
<blockquote
cite="mid:d24863920903222250g7f755039m9cefe643d48cc1f2@..."
type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">- AuFS is used as an overlay system and is required for the NFS-booted
nodes. One might want to migrate that to a less efficient tmpfs approach
(ref: LTSP implementation as being done under Gentoo too).
- Change most of the scripts to use dialog and automate some detection/ask
questions as per what was discovered.
- ... and so on, the idea would be to implement my original proposal and
push it further. I do have it but want some feedback before I dig it out and
re-post it (Donnie, thought you had posted it as an example? If not so, feel
free to do so if it's still relevant.)
</pre>
</blockquote>
<pre wrap=""><!---->
I have been working on building the liveCD the last few days, and I am
almost finished. I have run into many problems along the way, and I
have come up with a few general ideas for things to include in my
proposal:
-Add spec files for x86.
</pre>
</blockquote>
yes<br>
<blockquote
cite="mid:d24863920903222250g7f755039m9cefe643d48cc1f2@..."
type="cite">
<pre wrap="">-Update the hpc-clustering and nfsroot profiles and the catalyst spec files.
</pre>
</blockquote>
yes<br>
<blockquote
cite="mid:d24863920903222250g7f755039m9cefe643d48cc1f2@..."
type="cite">
<pre wrap="">-Simplify and automate the build process with a configuration script.
</pre>
</blockquote>
? Meaning that you'd make a script that calls catalyst? Be my guest ;)<br>
<blockquote
cite="mid:d24863920903222250g7f755039m9cefe643d48cc1f2@..."
type="cite">
<pre wrap="">-Add documentation for building the liveCD using the new configuration script.
</pre>
</blockquote>
Or liveMedium as I started calling it for USB usage (or external HDD
;). Documentation is _always_ a good thing.<br>
<blockquote
cite="mid:d24863920903222250g7f755039m9cefe643d48cc1f2@..."
type="cite">
<pre wrap="">
These tasks all involve the build process, and once they have been
completed, work could begin on USB booting, JFFS2 or other areas of
interest.
</pre>
</blockquote>
Booting off USB should be the priority, even before nice docs or
automating the calling of catalyst. I didn't make a script for building
the images since the script does very little once the environment is
set up.<br>
<blockquote
cite="mid:d24863920903222250g7f755039m9cefe643d48cc1f2@..."
type="cite">
<pre wrap="">
It seems like a lot of work has been done to make sure the right
packages are included in the build, and that currently the biggest
challenge is updating the build process and making it easy to
duplicate.</pre>
</blockquote>
I don't know the status of the new version of Catalyst but many new
features, such as building stages 1-3 or even 1-4 in a single call, are
supposed to be added. I suggest checking out the status of that project
and then planning the "move" if applicable. Otherwise we'll have to
stick to the hackerish scripts and catalyst specs.<br>
<br>
My notes on why duplicating the build process tend to point to the
portage tree being so dynamic and the stage builds being very sensitive
to the tree. With the advent of snapshots and stages being produced on
a regular basis, the stage build pains should be lessened significantly
compared to the quarterly or bi-annual releases.<br>
<blockquote
cite="mid:d24863920903222250g7f755039m9cefe643d48cc1f2@..."
type="cite">
<pre wrap=""> Will improving the build process, USB booting and the
related file system changes be enough work for one summer, or are
there other parts of the cluster liveCD that Gentoo developers would
like to see improved?
</pre>
</blockquote>
Most of the configuration script(s) should be modified into a dialog(1)
based script where the user is presented options that would be
selected. The setup script I made is quite crude and reads in 2 config
files. This should be changed to use dialog and provide the user the
opportunity to change the default values.<br>
<br>
Another significant leap would be to be able to boot the nodes off
NFSv4. This may sound trivial but the kernel deosn't support this
natively which implies using a ramdisk (initrd). NFSv4 has many
advantages:<br>
<br>
* Ability to use links outside of the exported root (server side):<br>
-- this means that most of the common config files between the
server and nodes could be links...very _interesting_<br>
* better caching<br>
* all sorts of new goodies I am forgetting about<br>
<br>
The problem with this might be that the overlayed filesystem used for
the nfsroot might not like NFSv4 (aufs is already having a hard time
with NFS at all). This might be yet another reason to use tmpfs+bind
mounts<b>¹</b> as being done for the Gentoo MueCow project.<br>
<br>
<blockquote
cite="mid:d24863920903222250g7f755039m9cefe643d48cc1f2@..."
type="cite">
<pre wrap="">-Tom Stellard
</pre>
</blockquote>
So, there is still much to be done OS side even before we start looking
at goodies such as integrating TAU² and other popular scientific
apps...which mostly also imply good X drivers with all the surrounding
issues... We might even have enough meat for more than one SoCer on
this :P.<br>
<br>
Eric Thibodeau<br>
<br>
¹ <a class="moz-txt-link-freetext" href="http://www.funtoo.org/en/articles/linux/ffg/3/">http://www.funtoo.org/en/articles/linux/ffg/3/</a> "Tmpfs and bind mounts"<br>
² <a class="moz-txt-link-freetext" href="http://www.cs.uoregon.edu/research/tau/home.php">http://www.cs.uoregon.edu/research/tau/home.php</a> <br>
<br>
</body>
</html>
|
