[gentoo-desktop] Re: disk partitioning

[Duncan]

Mikko Husari <husku@×××××.net> posted 45C82D9C.7010601@×××××.net,
excerpted below, on  Tue, 06 Feb 2007 09:26:20 +0200:

> hi!
>
> i was wonderin (also tried my luck on perfomance-gentoo, but no one
> home), what kind of partition + fs table would be optimal on server
> and/or desktop. afaik, /usr/portage would be on its own partition, and
> perhaps reiserfs and raid0. distfiles should be on a different
> partition, so it would not be in the way of portage itself... but, what
> about other parts of gentoo/linux. and is journaling filesystem over
> striping raid just asking for trouble?

I'm running a 300-gig/disk, 4-SATA-disk, kernel RAID here, and /love/ it!

Each of the four disks is partitioned identically to the others, with the
corresponding partitions on each grouped in the corresponding RAIDs.

/dev/md0 (on /dev/sd[a..d]1) is raid-1, necessary for boot, since that's
all grub and the like understand, but that's all that's on the raid-1.

The main system is partitioned (mdp) raid-6, /dev/md_d1 on /dev/sd[a..d]2.
I have it partitioned three-ways, but it would be four if I were doing it
over.  The first two RAID partitions (would be three), /dev/md_d1p1 and
md_d1p2, are identically sized, containing my main root filesystem and a
backup, rootbak.  (I'd create rootbak1 and rootbak2, now.)  /dev/md_d1p3
(would be p4 or possibly reserving that as extended, make it p5, if I had
a second rootbak image) is a huge partition, containing an LVM2 managed
physical volume that in turn contains all the critical non-root data I
still want to keep on raid-6.

This is critical to my strategy -- the kernel itself knows how to deal
with RAID, both partitioned and unpartitioned, with any necessary
parameters passable on the kernel command line from grub/lilo/whatever.
The same CANNOT be said about LVM2, which requires userspace configuration
to load. Thus my choice of partitioned RAID for my main root filesystem
and its backups.  I don't have to hassle an initramfs/initrd with just
RAID, as I'd have to do if I put root on LVM2.

Equally important are the root and rootbak images and how I manage them. 
First, a lesson I learned the hard way, root contains everything portage
installs to, so /usr (minus some subdirs such as /usr/local and /usr/src)
and /var (minus subdirs such as /var/log), as well as the stuff
traditionally on root.  The problem I ran into before is that I had my
/usr partition go bad on me, and the backup version of it I had was
somewhat outdated.  Thus, the portage database in /var/db/portage was in
sync with what was on root, but NOT in sync with what was on /usr, which
was some six months outdated.  Recovery was possible, but not easy.  Thus
my current strategy -- keep everywhere portage installs packages to on the
same partition, so it's always in sync.  If I end up resorting to a backup
image, the entire backup image may be outdated, but what's on the disk is
always going to be in sync with what's in the portage database on the same
partition.

Second, the rootbak images are simply snapshots of root, taken
periodically when I think the system is relatively stable.  I have one. 
As I said, if I were doing it again, I'd make a second one, alternating
backups, so if tragedy struck when I was actually doing the backup, I'd
have the other one still intact and ready to boot to.  It's important to
note that while this doesn't replace off-system backups, it does make up
for fat-fingering or the occasional botched upgrade -- and with it on
RAID-6, it's relatively hardware failure resistant as well. If I ever
can't boot my main root, I simply reboot and tell grub to pass rootbak to
the kernel instead of root.  What's great about this is that other than
being a bit outdated, I have a fully functional system on the backup
image, just as I left it when I did the snapshot.  No having to fiddle
with text-mode links/lynx to google for what went wrong in ordered to
recover my main system, as I have a full X and KDE -- and everything else
I had installed at the time, sitting there fully configured and ready to
run.  Even if LVM goes out on me, while I lose /home and the like as they
are on the LVM handled raid-6 partition, I have all the system executables
including everything on /usr ready to use to solve the problem, should it
be necessary to google or use email or the like.  (/home is handled by
LVM2, but I could create a temp /home and use it to google or whatever, if
necessary.)

So then as I mentioned, the last raid-6 partition is LVM2 managed and
contains logical volumes (which I still refer to as partitions) for all my
usual data, /home and a homebak, /var/log (I decided if it died I'd just
take the loss, so no bak of it, but it's at least on raid-6), a mail
partition and mailbak, news (no bak), a multimedia partition and bak,
/usr/local and a bak, ... .  With portage I run FEATURES=buildpkg, and
want it on the raid-6, so there's a package partition and its bak here as
well.

/dev/sd[a..d]3 is swap.  I have the kernel manage this directly, as with a
priority= parameter in the mount options set to the same thing (here,
priority=1 for all four partitions, one on each spindle), the kernel will
stripe swap as if it was a raid-0.  If I needed zero-downtime, I'd put
swap on the raid-6 as well, so it could continue to function if I lost up
to two disks, but as long as my data is safe, a forced reboot on a lost
disk isn't a big deal here, and the speed of striped is nice, so striped
it is!

/dev/sd[a..d]4 is reserved as the extended partition, thus giving me room
to expand beyond four partitions if need be.

/dev/sd[a..d]5 is striped/raid-0 (mdp/partitioned), containing stuff that
doesn't need the redundancy of the raid-6.  /tmp and /var/tmp of course
on one partition. The second contains the rest.  The portage and kernel
trees are easily redownloadable, so /usr/src and $PORTDIR live here.  I
run ccache, and figure the speed of striped is better than the redundancy
of raid-6, so it's here too. Everything except the tmps are on the same
partition, in subdirs, with symlinks from their normal location where
necessary (as with /usr/src).

Sizes?  The root images are ~10 gig.  The workspace for portage needs to
be about 5 gig in some cases.  That's normally /var/tmp but I have
/var/tmp symlinked to /tmp anyway so put it there and have a fairly large
(~25 gig) /tmp.  The other striped partition is ~12 gig.  The package
partitions are ~4 gig.  /var/log is ~2 gig.  /home (and its bak) are 10
gig, but I have additional dedicated partitions for some stuff people
would otherwise store there such as mail and multimedia.  Almost
everything is less than half full, with only 1.5 gig of the 10 gig root
partition and baks actually used, but I wanted to be sure and leave
/plenty/ of room there.

Filesystems?  Here, I'm running 100% reiserfs, thus avoiding having to
keep other types loaded (tho I build ext2 as a module, to use with
floppies if necessary, and of course iso9660 and udf for CD/DVD, again
as modules, only loaded when necessary). I like the reiserfs tail-packing,
and haven't had any problems with it since the patches adding data=ordered
by default to reiserfs. Some say it's a dead-end, but it's quite stable
now and due to the number of systems installed with it, will need to be
maintained in the kernel for many years to come.  I had intended to
eventually upgrade to reiser4, but now, who knows?  Anyway, I can't see
going to a block system that wastes all that space with small files, and
the other alternative that handles that, XFS AFAIK, has issues of its own,
particularly on systems without very sturdy UPSs and/or battery-backed 
SCSI. I figure reiserfs has another three years anyway, before it starts
getting crufty, and three years is a long time in kernel and filesystems
development time, so what developments and options there might be by then,
who knows?  (According to the article someone else linked on reiserfs,
globalfs looks to be the eventual replacement, but it's not ready yet.)
By that time, my SATA hardware will be pretty old as well, so it'll be
time to think about upgrades there too, and I'll probably do the
filesystem upgrade if I think I need it, at the hardware upgrade.

I honestly can't say what filesystem I'd recommend now, but both ext3
(upgrading to 4 when appropriate) and reiserfs should be safe and stable
for years to come.  There's no reason to shy away from reiserfs now,
particularly if you already know and are comfortable with it (it works
just fine on RAID).  The one thing that CAN be said, however, is that of
all the available filesystems, the ext* series is the one best understood
by the kernel hackers in general.  The others have a limited subset that
understand them, but ext2/3/4/... is what Linus and Andrew and the other
"core" kernel hackers know best, and thus in some ways could be considered
to be the best supported.  In that regard, none of the others can equal
ext*, despite the fact that its base isn't as modern in some ways as some
of the other choices, and it's more modern in the way it integrates with
the kernel anyway, just because all the kernel hackers know it and thus do
the updates to it right away, that the others have to wait for their
groups to pickup on.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-desktop@g.o mailing list