Re: [gentoo-dev] [pre-GLEP] Repository configuration file (layout.conf)

[Michał Górny]

On Wed, 2021-05-19 at 10:37 -0400, Wolfgang E. Sanyer wrote:
> See some comments in-line below.
>
>
> On Wed, May 19, 2021 at 8:33 AM Michał Górny <mgorny@g.o> wrote:
> >
> > Hi,
> >
> > Please review the pre-GLEP inlined below.  Its purpose is to formally
> > define the format of layout.conf.  It's pretty much inevitable these
> > days, so we should specify it.  However, it doesn't really fit into PMS,
> > and other formats (Manifests, metadata.xml) are already defined
> > in GLEPs, so this just follows suit.
> >
> > Pre-GLEP follows.
> >
> > ---
> > GLEP: 9999
> > Title: Repository configuration file (layout.conf)
> > Author: Michał Górny <mgorny@g.o>
> > Type: Standards Track
> > Status: Draft
> > Version: 1.0
> > Created: 2021-05-19
> > Last-Modified: 2021-05-19
> > Post-History: 2021-05-19
> > Content-Type: text/x-rst
> > ---
> >
> > Abstract
> > ========
> >
> > The ``metadata/layout.conf`` file format is specified as used by Portage
> > and PkgCore.  A standard set of configuration keys is described
> > including the keys currently used in the Gentoo repository.
> >
> >
> > Motivation
> > ==========
> >
> > The ``metadata/layout.conf`` file was first added to the Gentoo
> > repository in Oct 2011, to facilitate setting of hashes used
> > in Manifest2 files.  In Mar 2012, it was used to indicate the transition
> > to the new ``md5-dict`` cache format.  In Jul 2013, it started being
> > used to indicate the repository's masters and effectively became
> > obligatory for all repositories.
> >
> > Today, ``layout.conf`` is used for various repository configuration
> > knobs that can be expressed as simple values and therefore
> > do not justify adding new files to the repository.  This primarily
> > involves the configuration of development tools but also includes a few
> > keys relevant to the behavior of the package manager.
> >
> > However, ``layout.conf`` is currently not covered by any formal
> > specification.  The PMS neglects its existence entirely, and the keys
> > used are roughly defined by their first use of Portage or PkgCore.
> > This GLEP aims to overcome this by providing a formal specification
> > for the file, as well as an up-to-date list of permitted configuration
> > keys.
> >
> >
> > Specification
> > =============
> >
> > layout.conf file format
> > -----------------------
> >
> > Every ebuild repository must contain a ``metadata/layout.conf`` file.
> > The file uses a line-oriented text format.  Lines starting with ``#``
> > represent comments and are ignored, as are lines consisting entirely
> > of whitespace.  The remaining lines must contain a key followed
> > by equals sign (``=``), followed by a (possibly empty) value.  Each of
> > these elements can be surrounded by additional whitespace that
> > is stripped.
>
> Is a space around the "=" optional? I see it used below, but here it sounds
> like there should be no space.

The last sentence says you can add extra space and it will be stripped.
If you can think of a better way of wording that (I really wanted to
avoid 'optional whitespace, followed by key, followed by optional
whitespace... ' ;-).

>
> > Configuration keys
> > ------------------
> >
> > The ``layout.conf`` file must contain the ``masters`` key.  Other keys
> > listed in this specification are entirely optional.  The package
> > managers may choose to implement a subset of listed keys.  Unknown keys
> > must be ignored.
> >
> > The following keys are currently defined:
>
> Is the intent for the GLEP to define and specify every possible key-value
> combination? If not, then perhaps this section should be truncated a bit? i.e.
> "here are some examples of currently defined keys" and then list a handful.
>
> Additionally, maybe the spec should include some sort of requirement for
> documentation of variables, similar to what you've done below.
>
> Edit: ok, nevermind I see your recommendation at the bottom. But still,
> I would probably either (1) insist that new keys be added to this GLEP, or
> (2) insist that new keys are documented properly.

Let's put it like this: the goal is to document all the useful keys.
I can live with some ancient-not-really-useful keys not being documented
(e.g. Portage has some stale/redundant Manifest-related keys, I think).

>
> > masters = <space-separated values>
> >   Specifies the master repositories of this repository.  For stand-alone
> >   repositories, this must be set to an empty value.  Otherwise, it can
> >   list one or more repositories, separated by spaces.  This key must
> >   be specified.
>
> I realize this is the only compulsory key, but perhaps it should still go in
> a separate section in order to ensure that it stands out? i.e. "The following
> keys are mandory..." and then later "the following optional keys are
> currently defined"

I'm not sure, to be honest.  I didn't really want to add a section for
one key ;-).

>
> > manifest-hashes = <space-separated values>
> >   Specifies the list of hashes that should be used for new distfiles
> >   in the Manifest files.  The development tools may create a subset
> >   of the specified hashes if it is not updating the checksums for
> >   the specified distfile, or does not support the hash in question.
> >   The hash names are specified in GLEP 74.  [#GLEP74]_  The default
> >   set of hashes is implementation-defined.
> >
> > manifest-required-hashes = <space-separated values>
> >   Specifies the list of hashes that must be used in Manifest files.
> >   The development tools must support all the hashes listed there,
> >   and update distfile checksums to use these hashes (refetching
> >   if necessary).  This must be a subset of manifest-hashes.  If not
> >   specified, all hashes from manifest-hashes (or the default set)
> >   are considered required.
> >
> > use-manifests = ``strict``, ``true`` or ``false``
> >   Indicates the policy for creating and using Manifest files.  If set
> >   to ``strict``, Manifest files are created and files are required to
> >   match digests found in Manifests.  If set to ``true``, Manifests
> >   are created but digest mismatches are ignored.  If set to ``false``,
> >   Manifests are not used at all.  The default is ``strict``.
> >
> > update-changelog = ``true`` or ``false``
> >   Indicates whether the development tools should write ChangeLog files.
> >   The default is ``false``.
> >
> > cache-formats = <space-separated values>
> >   Specifies one or more cache formats used by the repository.
> >   The currently defined values are ``pms`` for the original format
> >   specified in the PMS and ``md5-dict`` for the md5-dict format
> >   introduced in Portage 2.2.0_alpha68.  The default is
> >   implementation-defined.
> >
> > eapis-deprecated = <space-separated values>
> >   Specifies one or more EAPIs that are to be considered deprecated
> >   by the development tools for use in ebuilds, i.e. their use should
> >   trigger a warning.  If not specified, no EAPIs are deprecated.
> >
> > eapis-banned = <space-separated values>
> >   Specifies one or more EAPIs that are to be considered banned
> >   by the development tools for use in ebuilds, i.e. their use should
> >   be blocked.  If not specified, no EAPIs are banned.
> >
> > repo-name = <string>
> >   Specifies the repository name.  If specified, it must be equal
> >   to the contents of ``profiles/repo_name``.  If not specified,
> >   it defaults to the same value.  Discouraged.
> >
> > aliases = <space-separated values>
> >   Specified one or more additional names that can be used to reference
> >   the repository (e.g. in repository dependencies).  If not specified,
> >   no aliases are defined.
> >
> > thin-manifests = ``true`` or ``false``
> >   If enabled, Manifest files in the package directory must contain only
> >   ``DIST`` entries.  If disabled, Manifest files in the package
> >   directory must list digests for all files found in the package
> >   directory and the files directory.  The default is ``false``.
> >
> > sign-commits = ``true`` or ``false``
> >   Indicates whether git commits are to be signed (using ``git commit
> >   --gpg-sign``.  The default is ``false``.
> >
> > sign-manifests = ``true`` or ``false``
> >   Indicates whether individual package Manifests should be PGP-signed.
> >   Note that this refers to the historical behavior of signing individual
> >   Manifests, not the GLEP 74 behavior of signing the top-level Manifest.
> >   [#GLEP74]_  The default is ``true`` if PGP signing is configured.
> >
> > properties-allowed = <space-separated values>
> >   Specifies the list of ``PROPERTIES`` tokens that are permitted
> >   to be used in ebuilds.  If present, the development tools should issue
> >   a warning if ``PROPERTIES`` contains any tokens that are not listed
> >   here.  If not specified, all tokens are permitted.
> >
> > restrict-allowed = <space-separated values>
> >   Same as properties-allowed, except for ``RESTRICT``.
> >
> > profile-formats = <space-separated values>
> >   Specifies the format used by profiles and/or extensions to it.
> >   The default is ``pms`` indicating the format specified in the PMS.
> >   Other values are implementation-defined.
> >
> >
> > It is recommended that any future keys are added to this GLEP before
> > being implemented.
> >
> >
> > Example
> > -------
> >
> > The following is an example configuration for a git repository with
> > Gentoo set as a master::
> >
> >     masters = gentoo
> >
> >     # git: do not use ChangeLog, use thin, unsigned Manifests
> >     update-changelog = false
> >     thin-manifests = true
> >     sign-manifests = false
> >
> >     # force the new md5-dict cache format
> >     cache-formats = md5-dict
> >
> >
> > Rationale
> > =========
> >
> > This GLEP is written almost 10 years after ``layout.conf`` was
> > originally introduced.  This made it necessary to write it in such a way
> > that both the modern and historical implementations in Portage
> > and PkgCore, as well as the use in the Gentoo repository
> > and a reasonably large subset of the other repositories would remain
> > compliant.
> >
> > The historical default of assuming ``masters = gentoo`` when unspecified
> > is omitted as it is not portable and verbosely deprecated for many
> > years in Portage.  All repositories are required to explicitly specify
> > their masters, or an empty value if they are stand-alone.
> >
> > The default for Manifest hashes and cache formats are left to be
> > implementation-defined, as the defaults changed over time and do not
> > match between package managers.  In particular, Portage attempts to
> > autodetect the cache format currently used in a given repository.
> >
> > The repo-name key has been originally added as an alternative to
> > ``profiles/repo_name``.  However, the latter file is still required
> > for PMS compliance.  Furthermore, given that it is much easier to parse,
> > there seems to be no appealing reason to work towards replacing that
> > file.  This means that for all practical reasons, the repo-name key
> > is redundant and is listed here for completeness only.
> >
> > The profile-formats key has been introduced to permit Portage-specific
> > extensions to the profile directory without having to introduce custom
> > EAPIs.  The exact extensions are considered outside the scope of this
> > specification.
> >
> >
> > Backwards Compatibility
> > =======================
> >
> > The existing implementations found in Portage and PkgCore conform
> > to this specification, so does the ``metadata/layout.conf`` file
> > found in the Gentoo repository.
> >
> >
> > Reference Implementation
> > ========================
> >
> > The support for ``metadata/layout.conf`` is already a part of Portage
> > and PkgCore.
> >
> >
> > References
> > ==========
> >
> > .. [#GLEP74] GLEP 74: Full-tree verification using Manifest files
> >    (https://www.gentoo.org/glep/glep-0074.html)
> >
> >
> > Copyright
> > =========
> >
> > This work is licensed under the Creative Commons Attribution-ShareAlike
> > 4.0
> > International License. To view a copy of this license, visit
> > https://creativecommons.org/licenses/by-sa/4.0/.
> >
> >
> > --
> > Best regards,
> > Michał Górny
> >
> >
> >
>

-- 
Best regards,
Michał Górny