Re: [gentoo-user] Re: distfiles contains extra files?

[Alan McKinnon]

On 13/08/2014 18:46, James wrote:
> Neil Bothwick <neil <at> digimed.co.uk> writes:
>
>
>>> I previously thought that /usr/portage/distfiles
>>> only contains  tar files. I have not clean up the
>>> system, as I'm moving (dupicating some files for my
>>> /usr/local/   needs.
>
>> It contains everything downloaded by ebuilds.
>
> I guess our ebuilds are getting creative on what they allow.....

ebuilds can do whatever they feel like doing, and there's no sane exact
policy on how to deal with the various sources they use (the word
sources used very loosely here).

The general idea is "small files go in the tree, bigger files are
downloaded to distfiles when the ebuild is emerged". Obviouslym, there's
no policy on what constitutes small and large

>
>>> I thought all patch files where kept in the subdirs where the
>>> ebuilds are located (files dir)? It is still where most *.patch
>>> files are located.

True, most patches are small. But there are 36,000 or so packages, what
if every one of them had 3 patches? That's 100,000 inodes right there
and every gentoo user must have all of them all of the time and they
must all be checked somehow with every --sync, and most user will only
use a small fraction of them. It makes more sense to download larger
patches when the ebuild is emerged. Yes, it makes fetch() take a few
seconds longer, but many other things are gained

>> ISTR the policy is to only include smaller patch files in the tree, large
>> patches are downloaded to avoid everyone having to sync them.
>
> OK, but wny not put them in a subdir under the specific ebuild
> requiring those smaller patches?   Seems to be a dumb move to me.

See above.

>
>>> *.exe  (like verdan32.exe webdin32.exe )
>> These are the installers for the MS corefonts.
>
> OK, why they need to be in /distfiles/? seriously .exe
> files in a repository for sources? ARE YOU KIDDING ME?
> Let's just ignor the /bin/sbin and name it /sbinge
> as devs must be 'binge drinking' if this is our standard?

Consider what those .exe files are - they are the "sources" for whatever
the ebuild needs to do, and they do not have to be .tar.gz files either.
Most .exes in distfiles are things like microsoft corefonts. Now these
are redistibutable, but only if the sources are not modified. the
sources are what microsoft published - a .exe file. The ebuild downloads
them, unpacks them, finds the .ttfs and installs them.

The fact that the packaging method used in this case is a .exe is
completely irrelevant - that is how upstream provides the sources and
ebuilds *always* store files fetch()ed to distfiles



>
> /bin/ms/    ?????  /usr/local/bin/    ????
>
>>> I guess I need "enlightenment" ?
>> You can use whichever DE you prefer :)
>
> U very funny! I use 'enlightenment' compliments to the
> Bhagavad Gita. I actually has to read the English version
> of the entire book and write a paper on it, in only 3 days.
> Academically speaking I got 'enlightenment' from that
> literarture class.......ymmv.
>
>>> cleaning up distfiles/ by hand?
>
>> You can use eclean to remove obsolete distfiles and packages. But don't
>> use eclean on a shared $DISTDIR.
>
> Yes, agreed. But the point is this sort of stuff belongs in the file
> sub-tree under the ebuild that requires it, not as part of /distfiles/
> imho. From my perspective strowing random files into /distfiles/
> puts it in the category of "cruft".

Nope. Considering the size and extent of the tree:

$ find /var/portage -type f | wc
 153047  153047 8404011
$ find /var/portage -type f -size -1024c | wc
  75350   75350 4304282
$ find /var/portage -type d | wc
  25920   25920  928148

Holy smoke Batman! One hundred and fifty three thousand files! Half of
them are smaller than one kilobyte!

I doubt you have more than 5000 or so patch files total for everything
installed, and that's being generous. Why download and sync all of that
with every sync? Makes more sense to just fetch them when you need them
and shove them in distfiles.

Remember the huge efforts a few years back to deal with the unique
characteristics of the tree? We were trying out reiserfs with and
without tail packing, comparing ext3 to ext2 and wondering if a journal
was needed, and much much more. These days ext4 deals with it so much
better but the fact still remains that the tree is a unique construct
and efforts to reduce it's size for things most poeple don;t use is
always a good thing




>
> Do you agree or disagree?
>
> James
>
>
>
>
>
>


-- 
Alan McKinnon
alan.mckinnon@×××××.com