Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

[Michał Górny]

W dniu sob, 27.01.2018 o godzinie 11∶36 +0000, użytkownik Roy Bamford
napisał:
> On 2018.01.27 08:30, Michał Górny wrote:
> > W dniu pią, 26.01.2018 o godzinie 20∶48 -0500, użytkownik Michael
> > Orlitzky napisał:
> > > On 01/26/2018 06:24 PM, Michał Górny wrote:
> > > >
> > > > The alternate option of using file hash has the advantage of
> >
> > having
> > > > a more balanced split.  Furthermore, since hashes are stored
> > > > in Manifests using them is zero-cost.  However, this solution has
> >
> > two
> > > > significant disadvantages:
> > > >
> > > > 1. The hash values are unknown for newly-downloaded distfiles, so
> > > >    ``repoman`` (or an equivalent tool) would have to use a
> >
> > temporary
> > > >    directory before locating the file in appropriate subdirectory.
> > > >
> > > > 2. User-provided distfiles (e.g. for fetch-restricted packages)
> >
> > with
> > > >    hash mismatches would be placed in the wrong subdirectory,
> > > >    potentially causing confusing errors.
> > > >
> > >
> > > The filename proposal sounds fine, so this is only academic, but:
> >
> > are
> > > these two points really disadvantages?
> > >
> > > What are we worried about in using a temporary directory? Copying
> >
> > across
> > > filesystem boundaries? Except in rare cases, $DISTDIR itself will be
> > > usable a temporary location (on the same filesystem), won't it?
> >
> > Why add the extra complexity when there's no need for one? Note that
> > there's also the problem of resuming transfers, so in the end we're
> > talking about permanent temporary directory where we keep unfinished
> > transfers.
> >
> > > For the second point, portage is going to tell me where to put the
> >
> > file,
> > > isn't it? Then no matter what garbage I download, won't portage look
> >
> > for
> > > it in the right place, because where-to-put-it is determined using
> >
> > the
> > > same manifest hash that determines where-to-find-it?
> >
> > No, it won't. Why would it? You're going to call something like:
> >
> >   edistadd foo.tar.gz bar.tar.gz
> >
> > ...and it will place the files in the right subdirectories.
> >
> > --
> > Best regards,
> > Michał Górny
> >
> >
> >
> >
>
> Michał,
>
> How does this work for fetch restricted files and finding other files
> no longer on the mirrors?
>
> Its no longer a download and move it to $DISTFILES, or is it?
> Whatever it is, users will need to do it unless files in  $DISTFILES
> are accepted by package managers if they are not found in the main
> structure.

I've just answered that, and it's in the GLEP also. There will be
a helper tool to make this easy. Furthermore, I think we may even make
Portage keep accepting both locations indefinitely.

As for finding files in your distdir, there's no reason why plain:

  find -name 'foo.tar.gz'

wouldn't work.

-- 
Best regards,
Michał Górny