Re: [gentoo-dev] New project: binhost

[Zac Medico]

On 2/23/21 12:33 PM, Zac Medico wrote:
> On 2/23/21 12:05 PM, Zac Medico wrote:
>> On 2/23/21 11:46 AM, Zac Medico wrote:
>>> On 2/20/21 8:17 PM, Zac Medico wrote:
>>>> IUSE_RUNTIME will obviously introduce conditionals in binary package
>>>> dependencies, but we should welcome these conditionals because they will
>>>> provide useful flexibility.
>>>>
>>>> I think IUSE_RUNTIME will be a very nice feature to have for project
>>>> binhost, since it will allow binary package dependencies to behave with
>>>> flexibility that more closely resembles the flexibility of ebuild
>>>> dependencies.
>>>
>>> We can borrow paludis's notion of pbins [1] to generate ebuilds which
>>> install pre-built content, and the generated ebuilds could have USE
>>> flags that behave similarly to IUSE_RUNTIME in the sense that changes to
>>> USE flags will result in different builds of pre-built content being
>>> installed. A content-hash distfiles layout [2] could serve as a
>>> convenient way to store separate builds of pre-built content for
>>> multiple combinations of USE flags, and a generated ebuild would index
>>> the build by USE flag combination.
>>>
>>> Also, for the generated ebuilds, we can generate USE flags to include
>>> separate SRC_URI downloads for pre-built content to support things like
>>> FEATURES=split-debug and FEATURES=install-sources.
>>
>> Note that all of this can use existing EAPI features, since everything new
>> would be implemented in an ebuild generator that generates a single
>> ebuild to index pre-built content from multiple binary package builds.
>>
>>> [1] https://paludis.exherbo.org/overview/pbins.html
>>> [2] https://bugs.gentoo.org/756778
>>
>
> For generated ebuilds, we'll have a choice to model things as USE flags
> or sub-packages. For example, content from FEATURES=split-debug and
> FEATURES=install-sources content might make more sense to model as
> sub-packages than USE flags. It makes more sense to generate a
> sub-package when there is a need for the sub-package to have USE flags.
> For example, a split-debug sub-package can have USE flags which index
> pre-built content from builds for multiple USE flag combinations.
> Similar USE flags could be useful for an install-sources sub-package if
> the source code has patches which are conditional on USE flags.

Since the generated ebuilds are inspired by pbins, we might call them
"ebins". Once we have designed an ebin generation process that we're
happy with, we should consider making it part of an EAPI, so that
package managers can generate "ebins" on request. The EAPI should
include ways to split out files and distribute them separately based on
USE flags and/or sub-packages, so that binhost users can easily filter
which files are installed based on USE configuration.

We can automatically map user's splitdebug and installsources FEATURES
settings into USE settings, in the same way that FEATURES=test
automatically maps to USE=test.

Each generated ebuild (ebin) will use its SRC_URI metadata to index
builds for each USE flag combination for which a build exists. For
example, for 3 USE flags, up to 8 combinations will be indexed:

IUSE="a b c installsources splitdebug"
SRC_URI="
  !a? !b? !c? ( mirror://binhost/24fe6bd377 )
  !a? !b?  c? ( mirror://binhost/fbe14cbb02 )
  !a?  b? !c? ( mirror://binhost/1dfff1f2ac )
  !a?  b?  c? ( mirror://binhost/ae60f2940d )
   a? !b? !c? ( mirror://binhost/2976e1acc0 )
   a? !b?  c? ( mirror://binhost/f4809db70c )
   a?  b? !c? ( mirror://binhost/ecd08466cf )
   a?  b?  c? ( mirror://binhost/0c00f33b2e )
  installsources? (
    !a? !b? !c? ( mirror://binhost/063a14d6c7 )
    !a? !b?  c? ( mirror://binhost/f67c311625 )
    !a?  b? !c? ( mirror://binhost/1dfff1f2ac )
    !a?  b?  c? ( mirror://binhost/17a673e16a )
     a? !b? !c? ( mirror://binhost/914d1cecfe )
     a? !b?  c? ( mirror://binhost/ca18d86a2b )
     a?  b? !c? ( mirror://binhost/6bce13471a )
     a?  b?  c? ( mirror://binhost/3a6bdcd228 )
  )
  splitdebug? (
    !a? !b? !c? ( mirror://binhost/29b2f38c41 )
    !a? !b?  c? ( mirror://binhost/8adc9bef51 )
    !a?  b? !c? ( mirror://binhost/954d2ce484 )
    !a?  b?  c? ( mirror://binhost/32a614aaca )
     a? !b? !c? ( mirror://binhost/3548a2302d )
     a? !b?  c? ( mirror://binhost/e0c02cdc88 )
     a?  b? !c? ( mirror://binhost/f9cbd3c181 )
     a?  b?  c? ( mirror://binhost/31d4c03474 )
  )
"

For installsources, we can automate deduplication, so that we can
distribute the same file content for multiple USE combinations when
appropriate. If all of the combinations have identical content, then it
will look like this:

  installsources? (
    !a? !b? !c? ( mirror://binhost/063a14d6c7 )
    !a? !b?  c? ( mirror://binhost/063a14d6c7 )
    !a?  b? !c? ( mirror://binhost/063a14d6c7 )
    !a?  b?  c? ( mirror://binhost/063a14d6c7 )
     a? !b? !c? ( mirror://binhost/063a14d6c7 )
     a? !b?  c? ( mirror://binhost/063a14d6c7 )
     a?  b? !c? ( mirror://binhost/063a14d6c7 )
     a?  b?  c? ( mirror://binhost/063a14d6c7 )
  )

In order to ensure that an ebin is not selected for a USE combination
that has not been built yet, combinations for existing builds will be
listed in REQUIRED_USE, like this:

REQUIRED_USE="
|| (
  ( !a !b !c )
  ( !a !b  c )
  ( !a  b !c )
  ( !a  b  c )
  (  a !b !c )
  (  a !b  c )
  (  a  b !c )
  (  a  b  c )
)
"

-- 
Thanks,
Zac