Re: [gentoo-dev] LFS QA warnings coming soon to a build near you

[Mike Frysinger]

On 03 Jun 2015 10:26, Alexis Ballier wrote:
> On Tue, 2 Jun 2015 10:13:54 -0400 Mike Frysinger wrote:
> > On 01 Jun 2015 10:15, Alexis Ballier wrote:
> > > On Sun, 31 May 2015 11:17:50 -0400 Mike Frysinger wrote:
> > > > (3) considering the glibc effort has been stalled for over a year,
> > > > (1) is something we can help accomplish and make reasonable
> > > > progress on (4) glibc isn't the only one that implements LFS in a
> > > > transparent backwards compatible manner
> > >
> > > maybe the fact that some file operations are broken with glibc's
> > > default settings on a somewhat popular fs would be a good argument
> > > to un-stall it ?
> >
> > it isn't that black & white.  while for many packages the change
> > would be harmless, glibc is more conservative than that.  it can be
> > shown that a not uncommon coding style is to use long's everywhere
> > when dealing with the fs. when sizeof(off_t)==sizeof(long), this
> > works out.  when off_t is suddenly twice as large, then you can get
> > truncation at best (which is kind of the status quo) and random
> > corruption at worse (like an API that takes pointers to off_t's but
> > the caller passed in a long).  if you want to join the list upstream
> > and contribute to the discussion, you're more than welcome to.  the
> > maintainers aren't clueless -- they fully understand the trade-offs
> > we're discussing here and know that 32bit fs accesses lead to
> > failures (including stat).
> >
> > since sitting on our hands and hoping for the best continues to
> > accomplish nothing, getting the various upstream packages to opt-in
> > to LFS themselves can make real progress now.
>
> agreed, thanks for the explanation
>
> i take it as the main goal of these qa warnings is to fix the above
> mentioned half broken packages and that pushing lfs flags everywhere is
> a slow but safe way to achieve it :)
>
> it'd be interesting to have data about how many packages can break;
> my belief is that more than half the c/c++ programs use one way or
> another fs ops but maybe less than one in a thousand will break; i can
> understand this is still too much for a libc (even though other
> 'breaking fixes' weren't so controversial in the past), and we'll see
> after gathering the data your qa warnings will provide

let's ignore the open/read/write issue for now on the assumption it's more 
uncommon to incidentally have that break.  instead, let's look at things that
can hit inodes -- stat & dir listing.  a file simply being allocated a 64bit
inode will break an app -- both directly stating and even trying to list the 
dir.  oops! :)

here's the command i'm using for just inode funcs:
	scanelf / -mqRygs \
		'^((__)?[fl]?x?stat(v?fs)?(at)?|alphasort|getdirentries|readdir(_r)?|scandir(at)?|versionsort)$' 
you'd have to ignore glibc and sandbox as they provide both 32bit & 64bit.

i have an x86 chroot for random testing, but it's not as "full" as my desktop.
it has a measily 245 packages installed.  of those:
 - 25 are using 32bit inode functions
 - 34 are using some 32bit fs interface (this is a superset of the above)

if someone had a fuller install of like >1000 pkgs, that'd be interesting.

> > as i already said, changing the default in *glibc* doesn't help
> > non-glibc systems.
>
> even with ten different libc's, the order of magnitude is still smaller
> for changing the default :)

true.  each fight though will be significantly higher i think, assuming you can 
even find someone to talk to (as not all are open source projects).  and it will 
take longer for the fix to trickle out to deployments.
-mike