Gentoo Archives: gentoo-embedded

From: Ed W <lists@××××××××××.com>
To: gentoo-embedded@l.g.o
Subject: Re: [gentoo-embedded] quickpkg and PKG_INSTALL_MASK
Date: Wed, 25 Jan 2012 14:05:40
In Reply to: Re: [gentoo-embedded] quickpkg and PKG_INSTALL_MASK by Todd Goodman
On 24/01/2012 14:06, Todd Goodman wrote:
> * Joakim Tjernlund<joakim.tjernlund@×××××××××.se> [120124 02:18]: >> Ed W<lists@××××××××××.com> wrote on 2012/01/23 19:43:49: > [ .. ] >>> I'm doing something like this using aufs. The performance seems "not >>> bad", but you get a couple MB or so memory hit. (I'm using squashfs as >>> well, so unsure which causes the main memory increase). >> hmm, not sure how aufs would work out. One would like to permanently delete the > aufs is A Union FileSystem. You can have a RO "branch" (say SquashFS) > and a r/w "branch" (say JFFS2 or UBIFS.) When you write it will write > to the r/w branch and when you read it will read from the r/w branch if > it exists, else the r/o. > > So, if you want to permanently delete the old software. Write it to the > filesystem r/w filesystem (initially or after you have the aufs mounts > mounted.) Then you can delete from the r/w filesystem and it's gone for > good.
Agreed. Just to augment that explanation: - The RO branch would be at the bottom (can be multiple of these) - The RW branch goes over the top (*can* be multiple of these, but one is more normal) - You can effectively delete stuff from the RO branches because Aufs has the concept of "white out" files. So for all intents and purposes the top RW layer can create any end result you like, including that of completely masking out some lower layer - With some knowledge of how the whiteout files work you can also "undo" changes to the RO files. Eg directly mounting the RW layer and erasing all files (from the RW layer) leaves you back with just the stacked RO layers again. I find this helpful for development where I can basically work live on the last released build and then by inspection the RW layer has all the changes needed to apply to the next RO layer release! I believe recent kernels also have a much simpler "Overlay Filesystem" that has fewer features. Also the big alternative to aufs is Unionfs 2 - most distros use aufs, but both seem viable? Finally note that you don't need to use aufs for the entire filesystem. A common setup might be to use a bunch of bind mounts where you know you don't need overlay features, eg /usr might be a bunch of overlays, /home might be a bind mount to writeable storage, /var might be a ram drive which is initialised from some fixed template, etc? In my case I have an overlay over most things, but /usr/lib/modules is a bind mount to a RO filesystem (you can't write to it), /home is mounted to my writeable storage (not layered), the main OS dirs are layered and /var is a mess...
> You still want to pick a r/w branch with a filesystem that handles power > cuts well. You can continue to use JFFS2.
Thought: Is there any evidence that one modern filesystem is better than another with regards to sudden power removal? You probably need to speak to filesystem experts at this point and define the kind of thing you are trying to protect against? Sounds like you have raw flash storage here, so that constrains your choices somewhat? Just note that with aufs you can use quite a few filesystems for the different layers. So for example you could have a base RW layer which is a DM snapshot, overlaid with a loopback mount to a DVD iso, overlay that with a squashfs, and finally overlay an Ext4 RW mount... (And of course each of the RO layers might be stored on varied filesystems themselves - check distributions such as Slax which allow you to overlay a squashfs that is itself inside some loopback mounted file...) I believe for most cheapo consumer flash storage where the underlying flash filesystem isn't exposed, its quite susceptable to *complete* failure with sudden poweroff? The issue is the invisible, underlying flash filesystem gets corrupted during a partial write and that can be the end of your flash drive - you don't even get to see it again to recover from it... I don't believe partitioning protects you from this, but of course separating read/write concerns to physically separate devices would help? I presume this isn't what you are using though? Good luck Ed W


Subject Author
Re: [gentoo-embedded] quickpkg and PKG_INSTALL_MASK Joakim Tjernlund <joakim.tjernlund@×××××××××.se>