Gentoo Archives: gentoo-dev

From: William Hubbs <williamh@g.o>
To: gentoo-dev@l.g.o
Cc: robbat2@g.o
Subject: Re: [gentoo-dev] [PATCH v2 1/4] eclass/go-module: add support for building based on go.sum
Date: Wed, 19 Feb 2020 05:46:54
Message-Id: 20200219054645.GA18470@linux1.home
In Reply to: [gentoo-dev] [PATCH v2 1/4] eclass/go-module: add support for building based on go.sum by "Robin H. Johnson"
1 On Mon, Feb 17, 2020 at 01:22:29AM -0800, Robin H. Johnson wrote:
2 > EGO_SUM mode now supplements the existing EGO_VENDOR mode.
3 >
4 > EGO_SUM should be populated by the maintainer, directly from the go.sum
5 > file of the root package. See eclass and conversion examples for further
6 > details: dev-go/go-tour, app-admin/kube-bench, dev-vcs/cli
7 >
8 > The go-module_set_globals function performs validation of
9 > inputs and dies on fatal errors.
10 >
11 > Signed-off-by: Robin H. Johnson <robbat2@g.o>
12 > ---
13 > eclass/go-module.eclass | 419 +++++++++++++++++++++++++++++++++++--
14 > profiles/thirdpartymirrors | 1 +
15 > 2 files changed, 397 insertions(+), 23 deletions(-)
16 >
17 > diff --git eclass/go-module.eclass eclass/go-module.eclass
18 > index 80ff2902b3ad..50aff92735af 100644
19 > --- eclass/go-module.eclass
20 > +++ eclass/go-module.eclass
21 > @@ -4,22 +4,45 @@
22 > # @ECLASS: go-module.eclass
23 > # @MAINTAINER:
24 > # William Hubbs <williamh@g.o>
25 > +# @AUTHOR:
26 > +# William Hubbs <williamh@g.o>
27 > +# Robin H. Johnson <robbat2@g.o>
28 > # @SUPPORTED_EAPIS: 7
29 > # @BLURB: basic eclass for building software written as go modules
30 > # @DESCRIPTION:
31 > -# This eclass provides basic settings and functions
32 > -# needed by all software written in the go programming language that uses
33 > -# go modules.
34 > +# This eclass provides basic settings and functions needed by all software
35 > +# written in the go programming language that uses go modules.
36 > +#
37 > +# You might know the software you are packaging uses modules because
38 > +# it has files named go.sum and go.mod in its top-level source directory.
39 > +# If it does not have these files, try use the golang-* eclasses FIRST!
40 > +# There ARE legacy Golang packages that use external modules with none of
41 > +# go.mod, go.sum, vendor/ that can use this eclass regardless.
42 > +#
43 > +# Guidelines for usage:
44 > +# "vendor/":
45 > +# - pre-vendored package. Do NOT set EGO_SUM or EGO_VENDOR.
46 > #
47 > -# You will know the software you are packaging uses modules because
48 > -# it will have files named go.sum and go.mod in its top-level source
49 > -# directory. If it does not have these files, use the golang-* eclasses.
50 > +# "go.mod" && "go.sum":
51 > +# - Populate EGO_SUM with entries from go.sum
52 > +# - Append license:${GENTOO_LICENSE} to any modules needed at runtime.
53 > +# dev-go/golicense can tell you which modules in a Golang binary are used at
54 > +# runtime (requires network connectivity).
55 > #
56 > -# If it has these files and a directory named vendor in its top-level
57 > -# source directory, you only need to inherit the eclass since upstream
58 > -# is vendoring the dependencies.
59 > +# None of the above:
60 > +# - Did you try golang-* eclasses first? Upstream has undeclared dependencies
61 > +# (perhaps really old source). You can use either EGO_SUM or EGO_VENDOR.
62 > +
63 > +#
64 > +# If it has these files AND a directory named "vendor" in its top-level source
65 > +# directory, you only need to inherit the eclass since upstream has already
66 > +# vendored the dependencies.
67 > +
68 > +# If it does not have a vendor directory, you should use the EGO_SUM
69 > +# variable and the go-module_gosum_uris function as shown in the
70 > +# example below to handle dependencies.
71 > #
72 > -# If it does not have a vendor directory, you should use the EGO_VENDOR
73 > +# Alternatively, older versions of this eclass used the EGO_VENDOR
74 > # variable and the go-module_vendor_uris function as shown in the
75 > # example below to handle dependencies.
76
77 I think we can remove the example with EGO_VENDOR and
78 go-module_vendor_uris; we really don't want people to continue following
79 that example.
80
81 > @@ -28,6 +51,28 @@
82 > # dependencies. So please make sure it is accurate.
83 > #
84 > # @EXAMPLE:
85 > +# @CODE
86 > +#
87 > +# inherit go-module
88 > +#
89 > +# EGO_SUM=(
90 > +# "github.com/aybabtme/rgbterm v0.0.0-20170906152045-cc83f3b3ce59/go.mod h1:q/89r3U2H7sSsE2t6Kca0lfwTK8JdoNGS/yzM/4iH5I= license:BSD-2,MIT"
91 > +# "github.com/aybabtme/rgbterm v0.0.0-20170906152045-cc83f3b3ce59 h1:WWB576BN5zNSZc/M9d/10pqEx5VHNhaQ/yOVAkmj5Yo="
92 > +# )
93 > +# S="${WORKDIR}/${MY_P}"
94
95 The default setting of S should be fine for most ebuilds, so I don't
96 think we need this in the example.
97
98 > +# go-module_set_globals
99 > +#
100 > +# SRC_URI="https://github.com/example/${PN}/archive/v${PV}.tar.gz -> ${P}.tar.gz
101 > +# ${EGO_SUM_SRC_URI}"
102 > +#
103 > +# LICENSE="some-license ${EGO_SUM_LICENSES}"
104 > +#
105 > +# src_unpack() {
106 > +# unpack ${P}.tar.gz
107 > +# go-module_src_unpack
108 > +# }
109 +#
110 I don't think I would put an src_unpack() in the example.
111
112 > +# @CODE
113 > #
114 > # @CODE
115 > #
116 > @@ -35,7 +80,7 @@
117 > #
118 > # EGO_VENDOR=(
119 > # "github.com/xenolf/lego 6cac0ea7d8b28c889f709ec7fa92e92b82f490dd"
120 > -# "golang.org/x/crypto 453249f01cfeb54c3d549ddb75ff152ca243f9d8 github.com/golang/crypto"
121 > +# "golang.org/x/crypto 453249f01cfeb54c3d549ddb75ff152ca243f9d8 github.com/golang/crypto"
122 > # )
123 > #
124 > # SRC_URI="https://github.com/example/${PN}/archive/v${PV}.tar.gz -> ${P}.tar.gz
125 > @@ -64,10 +109,12 @@ export GO111MODULE=on
126 > export GOCACHE="${T}/go-build"
127 >
128 > # The following go flags should be used for all builds.
129 > -# -mod=vendor stopps downloading of dependencies from the internet.
130 > # -v prints the names of packages as they are compiled
131 > # -x prints commands as they are executed
132 > -export GOFLAGS="-mod=vendor -v -x"
133 > +# -mod=vendor use the vendor directory instead of downloading dependencies
134 > +# -mod=readonly do not update go.mod/go.sum but fail if updates are needed
135 > +export GOFLAGS="-v -x -mod=readonly"
136 > +[[ ${#EGO_VENDOR[@]} -gt 0 ]] && GOFLAGS+=" -mod=vendor"
137 >
138 > # Do not complain about CFLAGS etc since go projects do not use them.
139 > QA_FLAGS_IGNORED='.*'
140 > @@ -77,6 +124,42 @@ RESTRICT="strip"
141 >
142 > EXPORT_FUNCTIONS src_unpack pkg_postinst
143 >
144 > +# @ECLASS-VARIABLE: EGO_SUM
145 > +# @DESCRIPTION:
146 > +# This is an array based on the go.sum content from inside the target package.
147 > +# Each array entry must be quoted and contain:
148 > +#
149 > +# "<module> <version> [h1:<hash>] [<key>:<value>]"
150 > +#
151 > +# To ease conversion from EGO_VENDOR, the presence of the h1:hash is NOT
152 > +# enforced in ebuilds at this time.
153 > +#
154 > +# The format is described upstream here:
155 > +# https://tip.golang.org/cmd/go/#hdr-Module_authentication_using_go_sum
156 > +#
157 > +# h1:<hash> is the Hash1 structure used by upstream Go
158 > +# Note that Hash1 is MORE stable than Gentoo distfile hashing, and upstream
159 > +# warns that it's conceptually possible for the Hash1 value to remain stable
160 > +# while the upstream zipfiles change. E.g. it does NOT capture mtime changes in
161 > +# files within a zipfile.
162 > +#
163 > +# <key>:<value> is an additional set of key-value tuples, to amend Gentoo
164 > +# metadata.
165 > +#
166 > +# The extra metadata keys accepted at this time are:
167 > +# - license: for dependencies built into the final runtime, the value field is
168 > +# a comma seperated list of Gentoo licenses to apply to the LICENSE variable,
169 > +#
170
171 There are two lines for each module in go.sum, the one with /go.mod as a
172 suffix to the version and the one without. We do not need both right?
173
174 > +# @EXAMPLE:
175 > +# # github.com/BurntSushi/toml is a build-time only dep
176 > +# # github.com/aybabtme/rgbterm is a runtime dep, annotated with licenses
177
178 I'm not sure we can distinguish between buildtime only and runtime deps.
179
180 > +# EGO_SUM=(
181 > +# "github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ="
182 > +# "github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU="
183 > +# "github.com/aybabtme/rgbterm v0.0.0-20170906152045-cc83f3b3ce59/go.mod h1:q/89r3U2H7sSsE2t6Kca0lfwTK8JdoNGS/yzM/4iH5I= license:BSD-2,MIT"
184 > +# "github.com/aybabtme/rgbterm v0.0.0-20170906152045-cc83f3b3ce59 h1:WWB576BN5zNSZc/M9d/10pqEx5VHNhaQ/yOVAkmj5Yo="
185 > +# )
186 > +
187 > # @ECLASS-VARIABLE: EGO_VENDOR
188 > # @DESCRIPTION:
189 > # This variable contains a list of vendored packages.
190 > @@ -95,10 +178,10 @@ go-module_vendor_uris() {
191 > local hash import line repo x
192 > for line in "${EGO_VENDOR[@]}"; do
193 > read -r import hash repo x <<< "${line}"
194 > - if [[ -n $x ]]; then
195 > + if [[ -n ${x} ]]; then
196 > eerror "Trailing information in EGO_VENDOR in ${P}.ebuild"
197 > eerror "${line}"
198 > - eerror "Trailing information is: \"$x\""
199 > + eerror "Trailing information is: \"${x}\""
200 > die "Invalid EGO_VENDOR format"
201 > fi
202 > : "${repo:=${import}}"
203 > @@ -106,18 +189,242 @@ go-module_vendor_uris() {
204 > done
205 > }
206 >
207 > +# @ECLASS-VARIABLE: _GOMODULE_GOPROXY_BASEURI
208 > +# @DESCRIPTION:
209 > +# Golang module proxy service to fetch module files from. Note that the module
210 > +# proxy generally verifies modules via the Hash1 code.
211 > +#
212 > +# Users in China may find some mirrors in the default list blocked, and should
213 > +# explicitly set an entry in /etc/portage/mirrors for goproxy to
214 > +# https://goproxy.cn/ or another mirror that is not blocked in China.
215 > +# See https://arslan.io/2019/08/02/why-you-should-use-a-go-module-proxy/ for further details
216 > +#
217 > +# This variable is NOT intended for user-level configuration of mirrors, but
218 > +# rather to cover Golang modules that might exist only on specific Goproxy
219 > +# servers for non-technical reasons.
220 > +#
221 > +# This variable should NOT be present in user-level configuration e.g.
222 > +# /etc/portage/make.conf, as it will violate metadata immutability!
223 > +: "${_GOMODULE_GOPROXY_BASEURI:=mirror://goproxy/}"
224
225 If this isn't supposed to be in user-level configuration, where should
226 it be set?
227
228 > +
229
230 > +# @ECLASS-VARIABLE: _GOMODULE_GOSUM_REVERSE_MAP
231 > +# @DESCRIPTION:
232 > +# Mapping back from Gentoo distfile name to upstream distfile path.
233 > +# Associative array to avoid O(N*M) performance when populating the GOPROXY
234 > +# directory structure.
235 > +declare -A -g _GOMODULE_GOSUM_REVERSE_MAP
236 > +
237 > +# @FUNCTION: go-module_set_globals
238 > +# @DESCRIPTION:
239 > +# Convert the information in EGO_SUM for other usage in the ebuild.
240 > +# - Populates EGO_SUM_SRC_URI that can be added to SRC_URI
241 > +# - Exports _GOMODULE_GOSUM_REVERSE_MAP which provides reverse mapping from
242 > +# distfile back to the relative part of SRC_URI, as needed for
243 > +# GOPROXY=file:///...
244 > +go-module_set_globals() {
245 > + local line exts
246 > + # for tracking go.sum errors
247 > + local error_in_gosum=0
248 > + local -a gosum_errorlines
249 > + # used make SRC_URI easier to read
250 > + local newline=$'\n'
251 > + # capture unique licenses
252 > + local -A unique_licenses
253 > +
254 > + # Now parse EGO_SUM
255 > + for line in "${EGO_SUM[@]}"; do
256 > + local module version modfile version_modfile kvs x
257 > + local hash1='' license=''
258 > + read -r module version_modfile kvs <<< "${line}"
259 > + # Parse the key:value set on the end of the line
260 > + for _kv in ${kvs}; do
261 > + local _v=${_kv#*:}
262 > + local _k=${_kv%$_v}
263 > + case "$_k" in
264 > + 'h1:') hash1+=" $_kv" ;;
265 > + 'license:') license+=" ${_v//,/ }" ;;
266 > + *) error_in_gosum=1 ; gosum_errorlines+=( "Unknown EGO_SUM key:value: k=$_k v=$_v" ) ;;
267 > + esac
268 > + done
269 > +
270 > + # Split 'v0.3.0/go.mod' into 'v0.3.0' and '/go.mod'
271 > + # It might NOT have the trailing /go.mod
272 > + IFS=/ read -r version modfile x <<<"${version_modfile}"
273 > + # Reject multiple slashes
274 > + if [[ -n ${x} ]]; then
275 > + error_in_gosum=1
276 > + gosum_errorlines+=( "Bad version: ${version_modfile}" )
277 > + continue
278 > + fi
279 > +
280 > + # The modfile variable should be either empty or '/go.mod'
281 > + # There is a chance that upstream Go might add something else here in
282 > + # future, and we should be prepared to capture it.
283 > + # The .info files do not need to be downloaded, they will be created
284 > + # based on the .mod file.
285 > + # See https://github.com/golang/go/issues/35922#issuecomment-584824275
286 > + exts=()
287 > + local errormsg=''
288 > + case "${modfile}" in
289 > + '') exts=( zip ) ;;
290 > + 'go.mod'|'/go.mod') exts=( mod ) ;;
291 > + *) errormsg="Unknown modfile: line='${line}', modfile='${modfile}'" ;;
292 > + esac
293 > +
294 > + # If it was a bad entry, restart the loop
295 > + if [[ -n ${errormsg} ]]; then
296 > + error_in_gosum=1
297 > + gosum_errorlines+=( "${errormsg} line='${line}', modfile='${modfile}'" )
298 > + continue
299 > + fi
300 > +
301 > + # Capture unique licenses
302 > + for _l in ${license} ; do
303 > + unique_licenses["${_l}"]=1
304 > + done
305 > +
306 > + _dir=$(_go-module_gomod_encode "${module}")
307 > +
308 > + for _ext in "${exts[@]}" ; do
309 > + # Relative URI within a GOPROXY for a file
310 > + _reluri="${_dir}/@v/${version}.${_ext}"
311 > + # SRC_URI: LHS entry
312 > + _uri="${_GOMODULE_GOPROXY_BASEURI}/${_reluri}"
313 > + # SRC_URI: RHS entry, encode any slash in the path as %2F in the filename
314 > + _distfile="${_reluri//\//%2F}"
315 > +
316 > + EGO_SUM_SRC_URI+=" ${_uri} -> ${_distfile}${newline}"
317 > + _GOMODULE_GOSUM_REVERSE_MAP["${_distfile}"]="${_reluri}"
318 > + done
319 > + done
320 > +
321 > + # Dedupe licenses
322 > + for license in "${!unique_licenses[@]}" ; do
323 > + EGO_SUM_LICENSES+=" ${license}"
324 > + done
325 > +
326 > +
327 > + if [[ ${error_in_gosum} != 0 ]]; then
328 > + eerror "Trailing information in EGO_SUM in ${P}.ebuild"
329 > + for line in "${gosum_errorlines[@]}" ; do
330 > + eerror "${line}"
331 > + done
332 > + die "Invalid EGO_SUM format"
333 > + fi
334 > +
335 > + # Ensure these variables not not changed past this point
336 > + readonly EGO_SUM
337 > + readonly EGO_SUM_SRC_URI
338 > + readonly EGO_SUM_LICENSES
339 > + readonly _GOMODULE_GOSUM_REVERSE_MAP
340 > +
341 > + # Set the guard that we are safe
342 > + _GO_MODULE_SET_GLOBALS_CALLED=1
343 > +}
344 > +
345 > +
346 > # @FUNCTION: go-module_src_unpack
347 > # @DESCRIPTION:
348 > +# Extract & configure Go modules for consumpations.
349 > +# - Modules listed in EGO_SUM are configured as a local GOPROXY via symlinks (fast!)
350 > +# - Modules listed in EGO_VENDOR are extracted to "${S}/vendor" (slow)
351 > +#
352 > +# This function does NOT unpack the base distfile of a Go-based package.
353 > +# While the entries in EGO_SUM will be listed in ${A}, they should NOT be
354 > +# unpacked, Go will directly consume the files, including zips.
355 > +go-module_src_unpack() {
356
357 If possible, this function should unpack the base distfile. That would
358 keep us from having to write src_unpack for every go ebuild that uses
359 the eclass.
360
361 > + if [[ "${#EGO_VENDOR[@]}" -gt 0 ]]; then
362 > + _go-module_src_unpack_vendor
363 > + elif [[ "${#EGO_SUM[@]}" -gt 0 ]]; then
364 > + _go-module_src_unpack_gosum
365 > + else
366 > + die "Neither EGO_SUM nor EGO_VENDOR are set!"
367
368 This shouldn't die, having neither one set is valid.
369
370 > + fi
371 > +}
372 > +
373 > +# @FUNCTION: _go-module_src_unpack_gosum
374 > +# @DESCRIPTION:
375 > +# Populate a GOPROXY directory hierarchy with distfiles from EGO_SUM
376 > +#
377 > +# Exports GOPROXY environment variable so that Go calls will source the
378 > +# directory correctly.
379 > +_go-module_src_unpack_gosum() {
380 > + # shellcheck disable=SC2120
381 > + debug-print-function "${FUNCNAME}" "$@"
382 > +
383 > + if [[ ! ${_GO_MODULE_SET_GLOBALS_CALLED} ]]; then
384 > + die "go-module_set_globals must be called in global scope"
385 > + fi
386 > +
387 > + local goproxy_dir="${T}/goproxy"
388 > + mkdir -p "${goproxy_dir}" || die
389 > +
390 > + # For each Golang module distfile, look up where it's supposed to go, and
391 > + # symlink into place.
392 > + local _A
393 > + local goproxy_mod_dir
394 > + for _A in ${A}; do
395 > + goproxy_mod_path="${_GOMODULE_GOSUM_REVERSE_MAP["${_A}"]}"
396 > + if [[ -n "${goproxy_mod_path}" ]]; then
397 > + debug-print-function "Populating goproxy for ${goproxy_mod_path}"
398 > + # Build symlink hierarchy
399 > + goproxy_mod_dir=$( dirname "${goproxy_dir}"/"${goproxy_mod_path}" )
400 > + mkdir -p "${goproxy_mod_dir}" || die
401 > + ln -sf "${DISTDIR}"/"${_A}" "${goproxy_dir}/${goproxy_mod_path}" || die "Failed to ln"
402 > + local v=${goproxy_mod_path}
403 > + v="${v%.mod}"
404 > + v="${v%.zip}"
405 > + v="${v//*\/}"
406 > + _go-module_gosum_synthesize_files "${goproxy_mod_dir}" "${v}"
407 > + fi
408 > + done
409 > + export GOPROXY="file://${goproxy_dir}"
410 > +
411 > + # Validate the gosum now
412 > + _go-module_src_unpack_verify_gosum
413 > +}
414 > +
415 > +# @FUNCTION: _go-module_gosum_synthesize_files
416 > +# @DESCRIPTION:
417 > +# Given a path & version, populate all Goproxy metadata files which aren't
418 > +# needed to be downloaded directly.
419 > +# - .../@v/${version}.info
420 > +# - .../@v/list
421 > +_go-module_gosum_synthesize_files() {
422 > + local target=$1
423 > + local version=$2
424 > + # 'go get' doesn't care about the hash of the .info files, they
425 > + # just need a 'version' element!
426 > + # This saves a download of a tiny file
427 > + # The .time key is omitted, because that is the time a module was added to the
428 > + # upstream goproxy, and not metadata about the module itself.
429 > + cat >"${target}/${version}.info" <<-EOF
430 > + {
431 > + "Version": "${version}",
432 > + "shortName": "${version}",
433 > + "Name": "${version}"
434 > + }
435 > + EOF
436 > + listfile="${target}"/list
437 > + if ! grep -sq -x -e "${version}" "${listfile}" 2>/dev/null; then
438 > + echo "${version}" >>"${listfile}"
439 > + fi
440 > +}
441 > +
442 > +# @FUNCTION: _go-module_src_unpack_vendor
443 > +# @DESCRIPTION:
444 > # Extract all archives in ${a} which are not nentioned in ${EGO_VENDOR}
445 > # to their usual locations then extract all archives mentioned in
446 > # ${EGO_VENDOR} to ${S}/vendor.
447 > -go-module_src_unpack() {
448 > - debug-print-function ${FUNCNAME} "$@"
449 > +_go-module_src_unpack_vendor() {
450 > + # shellcheck disable=SC2120
451 > + debug-print-function "${FUNCNAME}" "$@"
452
453 Maybe add an eqawarn here that EGO_VENDOR is deprecated to encourage
454 people to migrate their ebuilds.
455
456 > local f hash import line repo tarball vendor_tarballs x
457 > vendor_tarballs=()
458 > for line in "${EGO_VENDOR[@]}"; do
459 > read -r import hash repo x <<< "${line}"
460 > - if [[ -n $x ]]; then
461 > + if [[ -n ${x} ]]; then
462 > eerror "Trailing information in EGO_VENDOR in ${P}.ebuild"
463 > eerror "${line}"
464 > die "Invalid EGO_VENDOR format"
465 > @@ -125,10 +432,10 @@ go-module_src_unpack() {
466 > : "${repo:=${import}}"
467 > vendor_tarballs+=("${repo//\//-}-${hash}.tar.gz")
468 > done
469 > - for f in $A; do
470 > - [[ -n ${vendor_tarballs[*]} ]] && has "$f" "${vendor_tarballs[@]}" &&
471 > + for f in ${A}; do
472 > + [[ -n ${vendor_tarballs[*]} ]] && has "${f}" "${vendor_tarballs[@]}" &&
473 > continue
474 > - unpack "$f"
475 > + unpack "${f}"
476 > done
477 >
478 > [[ -z ${vendor_tarballs[*]} ]] && return
479 > @@ -145,13 +452,53 @@ go-module_src_unpack() {
480 > done
481 > }
482 >
483 > +# @FUNCTION: _go-module_src_unpack_verify_gosum
484 > +# @DESCRIPTION:
485 > +# Validate the Go modules declared by EGO_SUM are sufficent to cover building
486 > +# the package, without actually building it yet.
487 > +#
488 > +# This might be a good candidate for src_prepare in later versions of the
489 > +# eclass, but all consumers will need updating!
490 > +_go-module_src_unpack_verify_gosum() {
491 > + # shellcheck disable=SC2120
492 > + debug-print-function "${FUNCNAME}" "$@"
493 > +
494 > + if [[ ! ${_GO_MODULE_SET_GLOBALS_CALLED} ]]; then
495 > + die "go-module_set_globals must be called in global scope"
496 > + fi
497 > +
498 > + cd "${S}"
499 > +
500 > + # Cleanup the modules before starting anything else
501 > + # This will print 'downloading' messages, but it's accessing content from
502 > + # the $GOPROXY file:/// URL!
503 > + einfo "Tidying go.mod/go.sum"
504 > + go mod tidy >/dev/null
505 > +
506 > + # Verify that all needed modules are really present, by fetching everything
507 > + # in the package's main go.mod. If the EGO_SUM was missing an entry then
508 > + # 'go mod tidy' && 'go get' will flag it.
509 > + # -v = verbose
510 > + # -d = download only, don't install
511 > + # -mod readonly = treat modules as readonly source
512 > + einfo "Verifying linked Golang modules"
513 > + GO111MODULE=on \
514 > + go get \
515 > + -v \
516 > + -d \
517 > + -mod readonly \
518 > + all \
519 > + || die "Some module is missing, update EGO_SUM"
520 > +}
521
522 If EGO_SUM is up to date, this could also mean that upstream forgot to
523 run go mod tidy and commit the go.mod/go.sum before the release, so it
524 could be a bug that needs to be reported.
525
526 > +
527 > # @FUNCTION: go-module_live_vendor
528 > # @DESCRIPTION:
529 > # This function is used in live ebuilds to vendor the dependencies when
530 > # upstream doesn't vendor them.
531 > go-module_live_vendor() {
532 > - debug-print-function ${FUNCNAME} "$@"
533 > + debug-print-function "${FUNCNAME}" "$@"
534 >
535 > + # shellcheck disable=SC2086
536 > has live ${PROPERTIES} ||
537 > die "${FUNCNAME} only allowed in live ebuilds"
538 > [[ "${EBUILD_PHASE}" == unpack ]] ||
539 > @@ -168,7 +515,7 @@ go-module_live_vendor() {
540 > # @DESCRIPTION:
541 > # Display a warning about security updates for Go programs.
542 > go-module_pkg_postinst() {
543 > - debug-print-function ${FUNCNAME} "$@"
544 > + debug-print-function "${FUNCNAME}" "$@"
545 > [[ -n ${REPLACING_VERSIONS} ]] && return 0
546 > ewarn "${PN} is written in the Go programming language."
547 > ewarn "Since this language is statically linked, security"
548 > @@ -179,4 +526,30 @@ go-module_pkg_postinst() {
549 > ewarn "stable tree."
550 > }
551 >
552 > +# @FUNCTION: _go-module_gomod_encode
553 > +# @DESCRIPTION:
554 > +# Encode the name(path) of a Golang module in the format expected by Goproxy.
555 > +#
556 > +# Upper letters are replaced by their lowercase version with a '!' prefix.
557 > +#
558 > +_go-module_gomod_encode() {
559 > + ## Python:
560 > + # return re.sub('([A-Z]{1})', r'!\1', s).lower()
561 > +
562 > + ## Sed:
563 > + ## This uses GNU Sed extension \l to downcase the match
564 > + #echo "${module}" |sed 's,[A-Z],!\l&,g'
565 > + #
566 > + # Bash variant:
567 > + debug-print-function "${FUNCNAME}" "$@"
568 > + #local re input lower
569 > + re='(.*)([A-Z])(.*)'
570 > + input="${1}"
571 > + while [[ ${input} =~ ${re} ]]; do
572 > + lower='!'"${BASH_REMATCH[2],}"
573 > + input="${BASH_REMATCH[1]}${lower}${BASH_REMATCH[3]}"
574 > + done
575 > + echo "${input}"
576 > +}
577 > +
578 > fi
579 > diff --git profiles/thirdpartymirrors profiles/thirdpartymirrors
580 > index ad4c4b972146..d60f166e07c9 100644
581 > --- profiles/thirdpartymirrors
582 > +++ profiles/thirdpartymirrors
583 > @@ -25,3 +25,4 @@ sourceforge https://download.sourceforge.net
584 > sourceforge.jp http://iij.dl.sourceforge.jp https://osdn.dl.sourceforge.jp https://jaist.dl.sourceforge.jp
585 > ubuntu http://mirror.internode.on.net/pub/ubuntu/ubuntu/ https://mirror.tcc.wa.edu.au/ubuntu/ http://ubuntu.uni-klu.ac.at/ubuntu/ http://mirror.dhakacom.com/ubuntu-archive/ http://ubuntu.c3sl.ufpr.br/ubuntu/ http://ubuntu.uni-sofia.bg/ubuntu/ http://hr.archive.ubuntu.com/ubuntu/ http://cz.archive.ubuntu.com/ubuntu/ https://mirror.dkm.cz/ubuntu http://ftp.cvut.cz/ubuntu/ http://ftp.stw-bonn.de/ubuntu/ https://ftp-stud.hs-esslingen.de/ubuntu/ https://mirror.netcologne.de/ubuntu/ https://mirror.unej.ac.id/ubuntu/ http://kr.archive.ubuntu.com/ubuntu/ https://mirror.nforce.com/pub/linux/ubuntu/ http://mirror.amsiohosting.net/archive.ubuntu.com/ http://nl3.archive.ubuntu.com/ubuntu/ https://mirror.timeweb.ru/ubuntu/ http://ubuntu.mirror.su.se/ubuntu/ https://ftp.yzu.edu.tw/ubuntu/ https://mirror.aptus.co.tz/pub/ubuntuarchive/ https://ubuntu.volia.net/ubuntu-archive/ https://mirror.sax.uk.as61049.net/ubuntu/ https://mirror.pnl.gov/ubuntu/ http://mirror.cc.columbia.edu/pub/linux/ubuntu/archive/ https://mirrors.namecheap.com/ubuntu/
586 > vdr-developerorg http://projects.vdr-developer.org/attachments/download
587 > +goproxy https://proxy.golang.org/ https://goproxy.io/ https://gocenter.io/
588 > --
589 > 2.25.0
590 >
591 >

Attachments

File name MIME type
signature.asc application/pgp-signature

Replies