[gentoo-dev] [PATCH v2 0/4] pypi.eclass: Provenance verification API

[gentoo-dev] [PATCH v2 0/4] pypi.eclass: Provenance verification API

[Michał Górny]

Hi,

Here's another shot at provenance verification API.  To address your
concerns, I've renamed the flag from verify-sig to verify-provenance.
This should make the difference in semantics and security model clearer.


Michał Górny (4):
  pypi.eclass: Fix eclassdoc typo; <package> → <project>
  pypi.eclass: Update the @DESCRIPTION
  pypi.eclass: Introduce provenance verification API
  dev-python/pypi-attestations: Enable provenance verification

 dev-python/pypi-attestations/Manifest         |   1 +
 .../pypi-attestations-0.0.27.ebuild           |   1 +
 eclass/pypi.eclass                            | 135 ++++++++++++++++--
 profiles/arch/amd64/use.mask                  |   4 +
 profiles/arch/base/use.mask                   |   4 +
 5 files changed, 133 insertions(+), 12 deletions(-)

[gentoo-dev] [PATCH v2 1/4] pypi.eclass: Fix eclassdoc typo; <package> → <project>

[Michał Górny]

Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
 eclass/pypi.eclass | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/eclass/pypi.eclass b/eclass/pypi.eclass
index 47d40449d13a..c460d4618ad0 100644
--- a/eclass/pypi.eclass
+++ b/eclass/pypi.eclass
@@ -1,4 +1,4 @@
-# Copyright 2023-2024 Gentoo Authors
+# Copyright 2023-2025 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 # @ECLASS: pypi.eclass
@@ -161,7 +161,7 @@ _pypi_sdist_url() {
 # generated using build systems that did not follow PEP 625
 # (i.e. the sdist name contains uppercase letters, hyphens or dots).
 #
-# If <package> is unspecified, it defaults to ${PYPI_PN}.  The package
+# If <project> is unspecified, it defaults to ${PYPI_PN}.  The package
 # name is normalized according to the specification unless
 # `--no-normalize` is passed.
 #
@@ -183,7 +183,7 @@ pypi_sdist_url() {
 # @DESCRIPTION:
 # Output the wheel filename for the specified project/version tuple.
 #
-# If <package> is unspecified, it defaults to ${PYPI_PN}.  The package
+# If <project> is unspecified, it defaults to ${PYPI_PN}.  The package
 # name is normalized according to the wheel specification.
 #
 # If <version> is unspecified, it defaults to ${PV} translated
@@ -221,7 +221,7 @@ pypi_wheel_name() {
 # the wheel contents will be unpacked straight into ${WORKDIR}.
 # You need to add a BDEPEND on app-arch/unzip.
 #
-# If <package> is unspecified, it defaults to ${PYPI_PN}.
+# If <project> is unspecified, it defaults to ${PYPI_PN}.
 #
 # If <version> is unspecified, it defaults to ${PV} translated
 # via pypi_translate_version.  If it is specified, then it is used

[gentoo-dev] [PATCH v2 2/4] pypi.eclass: Update the @DESCRIPTION

[Michał Górny]

Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
 eclass/pypi.eclass | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/eclass/pypi.eclass b/eclass/pypi.eclass
index c460d4618ad0..75d15917a766 100644
--- a/eclass/pypi.eclass
+++ b/eclass/pypi.eclass
@@ -11,16 +11,18 @@
 # @DESCRIPTION:
 # The pypi.eclass can be used to easily obtain URLs for artifacts
 # uploaded to PyPI.org.  When inherited, the eclass defaults SRC_URI
-# and S to fetch .tar.gz sdist.  The project filename is normalized
-# by default (unless PYPI_NO_NORMALIZE is set prior to inheriting
-# the eclass), and the version is translated using
+# and S to fetch .tar.gz sdist.  The eclass defaults to using ${PN}
+# as PyPI project name; PYPI_PN can be set prior to inheriting it
+# to override that.  The source distribution filename is assumed
+# to be normalized by default; PYPI_NO_NORMALIZE can be set prior
+# to inherit to override that.  The version is translated using
 # pypi_translate_version.
 #
 # If necessary, SRC_URI and S can be overridden by the ebuild.  Two
 # helper functions, pypi_sdist_url and pypi_wheel_url are provided
 # to generate URLs to artifacts of specified type, with customizable
 # URL components.  Additionally, pypi_wheel_name can be used to generate
-# wheel filename.
+# the wheel filename.
 #
 # pypi_normalize_name can be used to normalize an arbitrary project name
 # according to sdist/wheel normalization rules.  pypi_translate_version

[gentoo-dev] [PATCH v2 3/4] pypi.eclass: Introduce provenance verification API

[Michał Górny]

Introduce a new API to verify provenance of PyPI artifacts.  To enable
it, set PYPI_VERIFY_REPO to the upstream repository URL.  The eclass
will automatically add a verify-provenance flag along with dependencies,
fetch the provenance file from PyPI and export src_unpack() to verify
it.

Support for provenance verification can be checked on PyPI's project
page.  If it is supported, the project metadata (i.e. "Project links")
is found in "Verified details", whereas otherwise only "Maintainers"
are in that section.  It can also be seen under "view details" for
individual artifacts.

The eclass also provides the low-level functions to account for special
needs: pypi_provenance_url and pypi_verify_provenance.

The bits are implemented directly in pypi.eclass rather than
verify-sig.eclass since they are pretty tightly bound to PyPI
infrastructure, with nontrivial URLs and a dedicated provenance file
format.  On top of that, due to a difference in semantics, the flag
is named verify-provenance rather than verify-sig.

Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
 eclass/pypi.eclass           | 119 +++++++++++++++++++++++++++++++++--
 profiles/arch/amd64/use.mask |   4 ++
 profiles/arch/base/use.mask  |   4 ++
 3 files changed, 122 insertions(+), 5 deletions(-)

diff --git a/eclass/pypi.eclass b/eclass/pypi.eclass
index 75d15917a766..38068aac3356 100644
--- a/eclass/pypi.eclass
+++ b/eclass/pypi.eclass
@@ -18,17 +18,24 @@
 # to inherit to override that.  The version is translated using
 # pypi_translate_version.
 #
-# If necessary, SRC_URI and S can be overridden by the ebuild.  Two
-# helper functions, pypi_sdist_url and pypi_wheel_url are provided
-# to generate URLs to artifacts of specified type, with customizable
-# URL components.  Additionally, pypi_wheel_name can be used to generate
-# the wheel filename.
+# If PYPI_VERIFY_REPO is set to a non-empty value, verify-provenance
+# flag is added along with necessary BDEPEND.  The provenance file
+# for the default source distribution is added to SRC_URI, and a default
+# src_unpack() is exported to verify its provenance.
+#
+# If necessary, SRC_URI and S can be overridden by the ebuild.  Three
+# helper functions, pypi_sdist_url, pypi_wheel_url
+# and pypi_provenance_url are provided to generate URLs to artifacts
+# of specified type, with customizable URL components.  Additionally,
+# pypi_wheel_name can be used to generate the wheel filename.
 #
 # pypi_normalize_name can be used to normalize an arbitrary project name
 # according to sdist/wheel normalization rules.  pypi_translate_version
 # can be used to translate a Gentoo version string into its PEP 440
 # equivalent.
 #
+# pypi_verify_provenance can be used to verify the provenance directly.
+#
 # @EXAMPLE:
 # @CODE
 # inherit pypi
@@ -65,6 +72,15 @@ _PYPI_ECLASS=1
 # @CODE
 : "${PYPI_PN:=${PN}}"
 
+# @ECLASS_VARIABLE: PYPI_VERIFY_REPO
+# @DEFAULT_UNSET
+# @PRE_INHERIT
+# @DESCRIPTION:
+# The repository to verify provenance against.  If set to a non-empty
+# value, the eclass will add a "verify-provenance" flag that can be used
+# to download the provenance for the distribution, and verify it
+# against the provenance and the specified repository.
+
 # @FUNCTION: _pypi_normalize_name
 # @INTERNAL
 # @USAGE: <name>
@@ -259,6 +275,76 @@ pypi_wheel_url() {
 	fi
 }
 
+# @FUNCTION: _pypi_provenance_url
+# @INTERNAL
+# @USAGE: <dist> [<project> [<version>]]
+# @DESCRIPTION:
+# Internal function to generate provenance URL for <dist>, using
+# specified <project> and <version>.  Returns the result
+# via _PYPI_ATTESTATION_URL.
+_pypi_provenance_url() {
+	if [[ ${#} -lt 1 || ${#} -gt 3 ]]; then
+		die "Usage: ${FUNCNAME} <dist> [<project> [<version>]]"
+	fi
+
+	local dist=${1}
+	local project=${2-"${PYPI_PN}"}
+	local version=${3-"$(pypi_translate_version "${PV}")"}
+	_PYPI_ATTESTATION_URL="https://pypi.org/integrity/${project}/v${version}/${dist}/provenance"
+}
+
+# @FUNCTION: pypi_provenance_url
+# @USAGE: <dist> [<project> [<version>]]
+# @DESCRIPTION:
+# Output the URL to PyPI provenance for the specified artifact.
+pypi_provenance_url() {
+	local _PYPI_PROVENANCE_URL
+	_pypi_provenance_url "${@}"
+	echo "${_PYPI_PROVENANCE_URL}"
+}
+
+# @FUNCTION: pypi_verify_provenance
+# @USAGE: <dist> <provenance> [<repo>]
+# @DESCRIPTION:
+# Verify the specified artifact's provenance.  <dist> is the path
+# to the artifact to verify, while <provenance> is the provenance file.
+#
+# The function defaults to using PYPI_VERIFY_REPO as the expected
+# repository URL.  This can be overridden by specfying <repo>.
+#
+# The function dies on verification failure.
+pypi_verify_provenance() {
+	if [[ ${#} -lt 2 || ${#} -gt 3 ]]; then
+		die "Usage: ${FUNCNAME} <dist> <provenance> [<repo>]"
+	fi
+
+	local dist=${1}
+	local provenance=${2}
+	local repo=${3-"${PYPI_VERIFY_REPO}"}
+
+	einfo "Verifying ${dist##*/} ..."
+	pypi-attestations verify pypi --offline \
+		--repository "${repo}" \
+		--provenance-file "${provenance}" \
+		"${dist}" ||
+		die "Provenance verification failed for ${dist##*/}"
+}
+
+# @FUNCTION: pypi_src_unpack
+# @DESCRIPTION:
+# A src_unpack implementation that verifies provenances.  Exported only
+# with PYPI_VERIFY_REPO.
+pypi_src_unpack() {
+	if use verify-provenance; then
+		local sdist_url=$(pypi_sdist_url)
+		local filename=${sdist_url##*/}
+
+		pypi_verify_provenance "${DISTDIR}/${filename}"{,.provenance}
+	fi
+
+	default
+}
+
 # @FUNCTION: _pypi_set_globals
 # @INTERNAL
 # @DESCRIPTION:
@@ -278,8 +364,31 @@ _pypi_set_globals() {
 	fi
 
 	SRC_URI=${_PYPI_SDIST_URL}
+
+	if [[ -n ${PYPI_VERIFY_REPO} ]]; then
+		local dist=${_PYPI_SDIST_URL##*/}
+		local _PYPI_ATTESTATION_URL
+		_pypi_provenance_url "${dist}" "${PYPI_PN}" \
+			"${_PYPI_TRANSLATED_VERSION}"
+
+		IUSE="verify-provenance"
+		SRC_URI+="
+			verify-provenance? (
+				${_PYPI_ATTESTATION_URL} -> ${dist}.provenance
+			)
+		"
+		BDEPEND="
+			verify-provenance? (
+				dev-python/pypi-attestations
+			)
+		"
+	fi
 }
 
 _pypi_set_globals
 
+if [[ -n ${PYPI_VERIFY_REPO} ]]; then
+	EXPORT_FUNCTIONS src_unpack
+fi
+
 fi
diff --git a/profiles/arch/amd64/use.mask b/profiles/arch/amd64/use.mask
index ede58b0fa8f2..6721b5ab73fb 100644
--- a/profiles/arch/amd64/use.mask
+++ b/profiles/arch/amd64/use.mask
@@ -4,6 +4,10 @@
 # Unmask the flag which corresponds to ARCH.
 -amd64
 
+# Michał Górny <mgorny@gentoo.org> (2025-09-02)
+# dev-python/pypi-attestations is keyworded here.
+-verify-provenance
+
 # Paul Zander <negril.nx+gentoo@gmail.com> (2025-05-01)
 # media-libs/libspng only keyworded on amd64, arm64
 -spng
diff --git a/profiles/arch/base/use.mask b/profiles/arch/base/use.mask
index 19a1851c5101..34b0aaacabc5 100644
--- a/profiles/arch/base/use.mask
+++ b/profiles/arch/base/use.mask
@@ -1,6 +1,10 @@
 # Copyright 1999-2025 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
+# Michał Górny <mgorny@gentoo.org> (2025-09-02)
+# dev-python/pypi-attestations is not keyworded everywhere.
+verify-provenance
+
 # Paul Zander <negril.nx+gentoo@gmail.com> (2025-05-01)
 # media-libs/libspng only keyworded on amd64, arm64
 spng

[gentoo-dev] [PATCH v2 4/4] dev-python/pypi-attestations: Enable provenance verification

[Michał Górny]

Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
 dev-python/pypi-attestations/Manifest                        | 1 +
 dev-python/pypi-attestations/pypi-attestations-0.0.27.ebuild | 1 +
 2 files changed, 2 insertions(+)

diff --git a/dev-python/pypi-attestations/Manifest b/dev-python/pypi-attestations/Manifest
index 1396b9a869b9..2d475693a137 100644
--- a/dev-python/pypi-attestations/Manifest
+++ b/dev-python/pypi-attestations/Manifest
@@ -1 +1,2 @@
 DIST pypi_attestations-0.0.27.tar.gz 123543 BLAKE2B 11e6dabcb58c5201af4df0392784c5cda02eabfde4d6d2cd39464d3bf89fa7a54495c435e86e5b7031ed7d9a85230ec273279d2f8ab86ba5e1dec2eb431829e4 SHA512 206c6aaf523a0143defe2ef35ee886d8d5d3cc7d3010a5f1ef6d5e9f0af3dac56b2a3b3af87256b50bf475d88ff397ea35f7512aac540a2918e8a52c91ac6e03
+DIST pypi_attestations-0.0.27.tar.gz.provenance 9602 BLAKE2B 70ace0f798e6bdeb1be3af64c78ba3d774a2944239776728c70c8d06f91c7a6fef1f929ca63ed52ffd83e4d623b1dfe457a670ab5c9e72908b59d93d1b54a3c5 SHA512 9cd9bed860bedd479800c52637e24a9fcbe9f03a8ea8c25a5c4029e4f8c61743843dbb583afd7958c6fa8418ef734f08d4a55a7c4222956fe1f934004c14cdf6
diff --git a/dev-python/pypi-attestations/pypi-attestations-0.0.27.ebuild b/dev-python/pypi-attestations/pypi-attestations-0.0.27.ebuild
index c19a7c9fe83a..bd6e00af8a18 100644
--- a/dev-python/pypi-attestations/pypi-attestations-0.0.27.ebuild
+++ b/dev-python/pypi-attestations/pypi-attestations-0.0.27.ebuild
@@ -4,6 +4,7 @@
 EAPI=8
 
 DISTUTILS_USE_PEP517=setuptools
+PYPI_VERIFY_REPO=https://github.com/trailofbits/pypi-attestations
 PYTHON_COMPAT=( python3_{11..13} )
 
 inherit distutils-r1 pypi

Re: [gentoo-dev] [PATCH v2 3/4] pypi.eclass: Introduce provenance verification API

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2412 bytes --]

On Tue, 2025-09-02 at 11:07 +0200, Michał Górny wrote:
> Introduce a new API to verify provenance of PyPI artifacts.  To enable
> it, set PYPI_VERIFY_REPO to the upstream repository URL.  The eclass
> will automatically add a verify-provenance flag along with dependencies,
> fetch the provenance file from PyPI and export src_unpack() to verify
> it.
> 
> Support for provenance verification can be checked on PyPI's project
> page.  If it is supported, the project metadata (i.e. "Project links")
> is found in "Verified details", whereas otherwise only "Maintainers"
> are in that section.  It can also be seen under "view details" for
> individual artifacts.
> 
> The eclass also provides the low-level functions to account for special
> needs: pypi_provenance_url and pypi_verify_provenance.
> 
> The bits are implemented directly in pypi.eclass rather than
> verify-sig.eclass since they are pretty tightly bound to PyPI
> infrastructure, with nontrivial URLs and a dedicated provenance file
> format.  On top of that, due to a difference in semantics, the flag
> is named verify-provenance rather than verify-sig.
> 
> Signed-off-by: Michał Górny <mgorny@gentoo.org>
> ---
>  eclass/pypi.eclass           | 119 +++++++++++++++++++++++++++++++++--
>  profiles/arch/amd64/use.mask |   4 ++
>  profiles/arch/base/use.mask  |   4 ++
>  3 files changed, 122 insertions(+), 5 deletions(-)
> 

Uh, forgot to add use.desc.  To avoid resending the whole series, here's
the missing bit:

diff --git a/profiles/use.desc b/profiles/use.desc
index e82c6e4e046c..dcf4bcd07544 100644
--- a/profiles/use.desc
+++ b/profiles/use.desc
@@ -363,6 +363,7 @@ valgrind - Enable annotations for accuracy. May slow down runtime slightly. Safe
 vanilla - Do not add extra patches which change default behaviour; DO NOT USE THIS ON A GLOBAL SCALE as the severity of the meaning changes drastically
 vcd - Video CD support
 vdpau - Enable the Video Decode and Presentation API for Unix acceleration interface
+verify-provenance - Verify provenance of PyPI distfiles (verifies that the artifact comes from upstream source repository)
 verify-sig - Verify upstream signatures on distfiles
 vhosts - Add support for installing web-based applications into a virtual-hosting environment
 videos - Install optional video files (used in some games)



-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [gentoo-dev] [PATCH v2 0/4] pypi.eclass: Provenance verification API

[Eli Schwartz]

[-- Attachment #1.1: Type: text/plain, Size: 1136 bytes --]

On 9/2/25 5:07 AM, Michał Górny wrote:
> Hi,
> 
> Here's another shot at provenance verification API.  To address your
> concerns, I've renamed the flag from verify-sig to verify-provenance.
> This should make the difference in semantics and security model clearer.


I really do not believe it helps at all to make it "clearer" that adding
the USE flag is harmful -- we should simply not add it at all.

And certainly I would rather it never be added to any packages I comaintain.

The USE flag description doesn't even make it clear that it doesn't
verify the code is the same as the upstream repository (e.g. due to
actions/cache poisoning it is not even possible to discover after the
fact that downloaded release dependencies were malicious).

If you want to verify this information a better mechanism would be to,
well, actually verify it. Download the github repository snapshot,
attempt to build an sdist, verify the bytes of all files in the sdist
are the same as the PyPI sdist. That would be something I'd find
actually interesting and useful and would love to personally use.


-- 
Eli Schwartz

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]