Re: [gentoo-dev] [EAPI 6] src_fetch() for fetching live sources

[Michał Górny]

Dnia 2013-08-27, o godz. 10:33:27
Ian Stakenvicius <axs@g.o> napisał(a):

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 27/08/13 06:01 AM, Michał Górny wrote:
> > Hello, all.
> >
> > As the next item for discussion in EAPI 6 I'd like to raise the
> > idea of introducing src_fetch() phase for fetching live sources.
> >
> > First of all, I'd like to emphasize that the additional phase
> > function will not make live ebuilds any more welcome than they are
> > now. They will still be discouraged, and subject to masking as
> > usual. However, the function will improve the experience of users
> > using them.
> >
> > The major goal is to add a src_fetch() phase that could be called
> > earlier during the ebuild process (e.g. within a parallel fetching
> > process) or in a 'emerge --fetchonly' or similar call.
> >
> > I will sum up the issues that came up already. However, feel free
> > to point out any more questions if you have some.
> >
>
> 0.  Why src_fetch() instead of pkg_fetch() ?  Technically this would
> be running based on the package and may not even need to touch ${S} if
> the implementation just handles getting the sources to a state where a
> src_unpack() can copy them over.  Being a pkg_* would also keep it out
> of the standbox (as i believe convention is that all src_* are
> supposed to be sandboxed, correct??)

I don't see why a binary package would need to fetch from VCS :).

Aside that, sandbox and other portage features can be controlled per
EAPI and per phase.

> > 1. How should we state fetch dependencies?
> >
> > Most of live ebuilds would need a dependency on the VCS they're
> > using. Currently, those dependencies are part of DEPEND. However,
> > if src_fetch() is done early or out-of-process, it will no longer
> > be good enough for us.
> >
> > I see two major directions here:
> >
> > a) explicitly stating fetch dependencies (as FDEPEND or using
> > magical USE flag thingie in DEPEND). Then, those dependencies would
> > be pulled in earlier than other DEPENDs and installed before
> > src_fetch() is called.
> >
> > I think this will require more work on Zac's side. Also, should we
> > then install VCS-es during 'emerge --fetchonly'?
> >
>
> This could be handled via support from the PM, too.  IE, if you want
> to be able to install git-based live ebuilds either install git by
> hand or have USE="git" enabled on portage/paludis/etc.

And the boundary between package manager and repo becomes blurry once
again. Please don't make eclasses for new VCS require update to portage
and vice versa.

> Either way, it does make sense to have the fetcher (and by extension,
> possibly also the unarchiver??) as a separate dependency type instead
> of being in DEPEND, since these would technically be packages that the
> PM itself needs now to do its thing rather than the software package.

What does having unpacker there technically give you? The only reason
for fetcher would be because fetch would be done much earlier than
unpack. Do you expect portage to unpack something and then go merge
remaining build-time deps? ;f

> > 2. Should src_unpack() be disallowed from fetching VCS sources?
> >
> > If we introduce src_fetch() in EAPI 6, then we should probably
> > expect all the live eclasses to use that rather than src_unpack()
> > for fetching. If we agree on this, we can extend portage's
> > network-sandbox to this phase, and likely filesystem sandbox as
> > well.
> >
>
> The portage network-sandbox feature is going to have to act in an
> EAPI-specific way -- ie, there's still plenty of EAPI5 and lower
> ebuilds that will inherit say, git-2.eclass and therefore that eclass
> will still need to support doing its bits in src_unpack.

Yep, there's no problem with implementing that.

> > 3. Is everyone ok with splitting VCS fetch into fetch+unpack?
> >
> > That is, the new design would be like:
> >
> > - src_fetch() fetches the updates from remote server to DISTDIR,
> >
> > - src_unpack() does the checkout from DISTDIR to S.
> >
> > I suspect that some live ebuilds may prefer different kind of
> > operation but this split seems quite reasonable.
>
> This implementation makes a lot of sense to me.  In terms of the
> specific implementation, I assume both src_fetch and src_unpack be
> imeplemented in i.e. git-2.eclass?  Or would there be an accompanying
> change to the default src_unpack behaviour to, say, '[[ -z $A ]] && cp
> - -r ${VCSDIR} ${S}' for EAPI6 ?

It will be implemented completely in eclass. There's no reason to make
EAPI more complex, especially that we can't really assume what VCS will
do. And not every live ebuild will actually use VCS.

-- 
Best regards,
Michał Górny