Re: [RFC] gpkg format proposal v2 (was: Re: [gentoo-portage-dev] [RFC] Improving Gentoo package format)

[Michał Górny]

On Sun, 2018-11-11 at 21:53 +0100, Michał Górny wrote:
> Hi,
>
> Ok, here's the second version integrating the feedback received.
> The format is much simpler, based on nested tarballs inspired by Debian.
>
> The outer tarball is uncompressed and uses '.gpkg.tar' suffix.  It
> contains (preferably in order but PM should also handle packages with
> mismatched order):
>
> 1. Optional (but recommended) "gpkg: ${PF}" package label that can be
> used to quickly distinguish Gentoo binpkgs from regular tarballs
> (for file(1)).
>
> 2. "metadata.tar${comp}" tarball containing binary package metadata
> as files.
>
> 3. Optional "metadata.tar${comp}.sig" containing detached signature
> for the metadata archive.
>
> 4. "contents.tar${comp}" tarball containing files to be installed.
>
> 5. Optional "contents.tar${comp}.sig" containing detached signature for
> the contents archive.
>
> Notes:
>
> a. ${comp} can be any compression format supported by binary packages.
> Technically, metadata and content archives may use different
> compression.  Either or both may be uncompressed as well.
>
> b. While signatures are optional, the PM should have a switch
> controlling whether to expect them, and fail hard if they're not present
> when expected.
>
>
> Advantages
> ----------
> Guaranteed:
>
> + The binary package is still one file, so can be fetched easily.
>
> + File format is trivial and can be extracted using tar(1) + compressor.
>
> + The metadata and contents are compressed independently, and so can be
> easily extracted or modified independently.
>
> + The package format provides for separate metadata and content
> signatures, so they can be verified independently.
>
> + Metadata can be compressed now.
>
> Achieved by regular archives (but might be broken if modified by user):
>
> + Easy recognition by magic(1).
>
> + The metadata archive (and its signature) is packed first, so it may be
> read without fetching the whole binpkg.
>
>
> Why not .ar format?
> -------------------
> The use of .ar format has been proposed, akin to Debian.  While
> the option is mostly feasible, and the simplicity of .ar format would
> reduce the outer size of binary packages, I think the format is simply
> too obscure.  It lives mostly as static library format, and the tooling
> for it is part of binutils.  LSB considers it deprecated.  While I don't
> see it going away anytime soon, I'd rather not rely on it in order to
> save a few KiB.
>
>
> Is there anything left to address?

Here's a quick & dirty xpak2gpkg converter:

https://gist.github.com/mgorny/cca78fb93f14aad11f43abe352caad06

It can be used to try out the format practically and flesh out
the details before we go for a formal spec.

-- 
Best regards,
Michał Górny