[gentoo-dev] Re: Diskless Gentoo client

[Pat Double]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Thanks for the suggestions. I added the "v3" option to the kernel so it uses 
NFS v3 protocol. I fixed my problem with X locking up. For some reason I had 
SMP and Preemption turned on in my client kernel. Not sure if SMP is the 
problem (but it is unnecessary), but preemption has been known not to play 
nice sometimes, I think it wasn't doing well with the NFS root. Anyhow, I got 
my X to start and login to the server, yeah! agetty still doesn't work, but I 
don't care :) Other than the kernel recompile, I left everything else the 
same.

Thanks for the suggestions.

On Saturday 14 December 2002 09:12 pm, Istvan Marko wrote:
> Pat Double <double@×××××××××.com> writes:
> > I'm having trouble setting up a diskless Gentoo client. I have a Gentoo
> > server that will serve the root filesystem via NFS.
>
> Pat,
>
> I have a Gentoo based netbooted setup that works quite well. In many
> ways it's similar to yours but I took a different approach for certain
> things. I don't have a solution for your specific problem but maybe
>
> you will find my notes below useful:
> > The client's root has been setup using a stage1 tarball and going
> > through the bootstrap, etc. process to get xfree merged. (The client
> > will only provide an XDM login into the server so I don't need a lot
> > of packages).
>
> Same here, except that I don't need X on the systems my setup is
> intended for so I haven't installed XFree86 (but I see no reason why
> it wouldn't work). I did the initial bootstrap using the chroot method
> (on a redhat box). I never mount my master tree directly on a live
> system's root, I use chroot whenever I need to emerge additional
> packages.
>
> > I've gotten far in the process: A correct kernel with NFS root support
> > has been setup. I've used the mknbi program to create a bootable network
> > image. I have a floppy disk that will grab the kernel from the server
> > using the etherboot package. A tftp server is setup correctly to server
> > the kernel. The kernel boots, finds the nfs server and mounts the correct
> > filesystem.
>
> I use PXE capable ethernet cards (Intel e100, 3com) and pxegrup (part
> of the grub package) to load the kernel. It is also possible to build
> a network enabled grub floppy if the ethernet cards used are not PXE
> capable. The advantage of using pxegrub is that you get the same
> flexibility and the menu system you get with normal grub.
>
> > It does go through the boot sequence. I have modified /sbin/rc to
> > create /etc, /var , /tmp and /root on tmpfs and copy the /etc and
> > /var files from the root filesystem into the temporary copy. I am
> > mounting /mnt/.etc-tmpfs to tmpfs, copying the files and then using
> > mount -o bind to mount /mnt/.etc-tmpfs to /etc. Same with /var. /tmp
> > and /root is directly mounted to tmpfs with no files in them.
>
> I took a different approach here. I keep the root filesystem on tmpfs
> and mount only /usr via NFS. I guess the simplest way to explain how
> this works is by going through the boot process:
>
> 1. BIOS loads pxegrub from the tftp server.
>
> 2. pxegrub loads the kernel image from tftp and NFS mounts a root
>    filesystem. I use the following kernel command line:
>
> kernel /bzImage ip=::::::autoconf root=/dev/nfs
> nfsroot=1.2.3.4:/linux-netboot/initfs,v3
>
> The root filesystem that gets NFS mounted here is minimal, its purpose
> is to setup the real root filesystem and mount /usr. It consists of a
> statically linked /bin/busybox binary, an /sbin/init shell script, an
> /etc/fstab (could do away with this), a couple device entries in /dev
> (/dev/console is needed) and the tar archive containing the template
> root filesystem. /bin/sh, echo, mkdir, mount and tar are all symlinks
> to the busybox binary. And there is a statically linked pivot_root
> binary, compiled from the util-linux sources.
>
> 3. The /sbin/init, a shell script, is started by the kernel.
>
> 4. This /sbin/init shell script creates a tmpfs filesystem under
>    /sysroot and populates it from the template tar archive.
>
> 5. The /sbin/init shell script NFS mounts /sysroot/usr, this will end
>    up as /usr after step 6.
>
> 6. Once /sysroot is populated the /sbin/init shell script uses
>    pivot_root to change the root file system over to
>    /sysroot. pivot_root is normally used by initrd setups but it works
>    great in this situation as well.
>
> 7. And finally /sbin/init uses chroot to invoke the real gentoo
>    /sbin/init
>
> 8. The regular gentoo /sbin/init and init scripts run performing the
>    normal startup. I have a line in /etc/conf.d/local.start to umount
>    /initfs (which is where the initial NFS root gets moved by
>    pivot_root)
>
>
> Here is the /sbin/init script I use:
>
> #! /bin/sh
> mount /proc
> echo "Mounting filesystems"
> mount /sysroot
> cd /sysroot
> echo "Extracting root"
> tar xzf /root.tar.gz
> cd /
> mount /sysroot/usr
> mkdir /sysroot/initfs
> cd /sysroot
> echo "done"
> umount /proc
> pivot_root . initfs
> exec chroot / /sbin/init
>
> and the corresponding /etc/fstab:
>
> proc /proc proc defaults
> /dev/shm /sysroot tmpfs defaults
> 1.2.3.4:/linux-netboot/usr /sysroot/usr nfs nfsvers=2,ro,nolock,nocto
>
>
> root.tar.gz contains the parts of the gentoo install that are not
> under /usr, in my setup this is about 17MB. This might seem a lot but
> keep in mind the memory used by tmpfs is swappable. If you are low on
> RAM and don't have a local disk to swap to you could reduce this by
> NFS mounting /bin and /lib in addition to /usr. I generate root.tar.gz
> with the following command:
>
> tar cvzf ../initfs/root.tar.gz --exclude './usr/*' --exclude='./var/db/*'
> --exclude='./var/tmp/portage/*' --exclude='./var/cache/*'
> --exclude='./var/log/*' --exclude='./mnt/.init.d/*' .
>
> > I have disabled the filesystem checks as well.
>
> One of my goals was to keep changes to the Gentoo init scripts
> minimal so I accomplished this by ln -s /bin/true /sbin/fsck.tmpfs
>
> [...]
>
> > Is there a known bug in the kernel or perhaps devfs? Should I not
> > being using devfs with /dev on nfs? I can move a copy of /dev to a
> > tmpfs if necessary, that's about the only thing I haven't tried.
>
> I am not sure what is causing the problems you described. devfs works
> well for me, keeping /dev on NFS on the other hand can be problematic,
> especially with NFS v2. With your setup you might need to ensure that
> no_root_squash (or the equivalent) is set in /etc/exports on the
> server and you could try adding ,v3 at the end of the nfsroot option
> in the kernel command line forcing the kernel mount of / to v3 (it
> uses v2 by default for some reason)
>
> Also, check out this thread on some NFS performance issues I hit on
> netbooted systems and the workaround:
> http://article.gmane.org/gmane.linux.nfs/1870
> (This is not directly related but you might come across it later)
>
> Let me know if you have any questions about the setup I use, there are
> probably many details I forgot about.
>
> Good luck,

- -- 
Pat Double, double@×××××××××.com
"In the beginning God created the heaven and the earth."
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE+ApNKdOmLNuoWoKgRAtLaAJ4g+Lxhx5ZA+NSZgolv+Hmd3nhplwCePX5x
YajdfL3Pp5SJuS56CWzhz5U=
=fc8x
-----END PGP SIGNATURE-----


--
gentoo-dev@g.o mailing list