Gentoo Archives: gentoo-dev

From: Rich Freeman <rich0@g.o>
To: gentoo-dev@l.g.o
Subject: Re: [gentoo-dev] Re: Bleeding edge hardened-sources: move PaX markings from ELF to Extended Attributes
Date: Thu, 08 Dec 2011 02:43:58
In Reply to: Re: [gentoo-dev] Re: Bleeding edge hardened-sources: move PaX markings from ELF to Extended Attributes by Mike Frysinger
On Wed, Dec 7, 2011 at 8:58 PM, Mike Frysinger <vapier@g.o> wrote:
> i have no sympathy for broken userspace code
I define broken userspace code as anything that uses fsync except for transactional synchronization with external sources. My system is a bit beefier now, but one of the biggest performance issues before the upgrade was that mythtv did an fsync all the time. What's the point in having RAM and buffer if some process is going to tell the kernel that it is worth taking a huge impact on everything else just to write one file a little faster. The irony in the end was that the result usually was dropped data in mythtv on another tuner since you'd end up with two processes stepping all over each other and the kernel probably could have easily flushed all that data if it were allowed to have more than a few MB of data in the cache between disk seeks. Filesystems need to fail gracefully. If I write data to disk and the power fails then the file should either contain what it did before the write, or the correct data after the write. As a compromise since COW is still new I'm fine with it even being partially modified, but things like zero-length files or other odd behavior is just dumb. And the correct userspace implementation is to call write, and close when you're done. I don't want every process on my system doing syncs just because some ext3 purist thinks that data is disposable by default, or in the magical world where every system is on a UPS, hardware never fails, and kernels never panic. I might live dangerously for a 50% performance gain, but when we're talking about shaving an extra 0.1% off of some benchmark then lets go ahead and write the data out in the right order. If we do want to have an API that tells the kernel that data isn't disposable or to safely overwrite a file in place then we should invent a new system call for that, and not try to overload some other behavior (like a combination of fsyncing and moves). The problem with the latter is that it often results in extra disk churn as the filesystem does more than is strictly necessary, and since every filesystem author has their own religious preference about the right series of system calls to use programs end up only working correctly on a handful of filesystems. And of course the real solution is COW once that becomes stable. And the next step after that is real transaction support. If only btrfs were ready for production use... Fortunately for the moment it seems Linus is reasonably sane and will fix filesystems if the upstream maintainers are unwilling to do so. :) Ok, off the soap box. My issue is that people seem to talk about "correct userspace behavior" as if there were some kind of standard we were referring to. By all means get the crazy behavior added to POSIX and then we can all go on and on about it (and then once COW comes along scrach our heads as we do all kinds of extra unnecessary file manipulation to do something that just works by default). If there is such a standard that has been endorsed by all the major unix filesystems, please do point it out. And sorry, Mike, not really directed at you in particular - I know that many people feel the same as you do... Rich