Re: [gentoo-dev] Current portage will now truncate your repo's git history to 1

[John Helmert III]

On Sat, Dec 17, 2022 at 05:42:43AM +0000, Sam James wrote:
>
>
> > On 15 Dec 2022, at 19:22, Florian Schmaus <flow@g.o> wrote:
> >
> > This is a public service announcement that the recently stabilized portage version will truncate you repo's git history to 1.
>
> I wish you'd shown us in #gentoo-portage before sending this, as it's a bit misleading / alarmist. We were actively all speaking
> at the time.
>
> Not only to reconsider the phrasing and include some important details, but also to mention the rationale, which I describe
> below.
>
> If you felt the Portage team should've written a news item for it, you were (and still are) free to say so, and we'd
> do it.
>
> >
> > While this is a good thing for the majority of Gentoo users, it affects developers that develop in a git known to portage (like me). If I understand the portage maintainers vision correctly, then future portage will assume full control over its configured repositories and potentially perform destructive operations on them, for example "git clean" [1].
> >
> ... only for repositories of sync-type=git & auto-sync=yes.
>
> The rationale for this is that most people who use sync-type=git are doing so because they want
> a quicker sync (to complete), a more reliable one (no Manifest race condition issues), and to
> get changes from Gentoo faster.
>
> Whenever Portage is syncing something, our view was that we should prioritise successful
> completion of syncs, especially given it may be performed unattended.
>
> If git is pulling from a CDN, Portage may on one sync receive state X, but
> on a subsequent sync receive state X-1. This can cause an odd state
> where git wants to resolve conflicts and manual intervention is required.
>
> This situation was often worse with sync-depth=1 and would lead
> to orphaned files, hence the 'git clean' PR referenced.
>
> The motivation behind the sync depth part was because of
> disk space growing unbounded otherwise.

Just to make this more abundantly explicit: it repeatedly happened to
me (but others, too) that on a system using git to sync a shallow
::gentoo clone, during unattended syncs, the repository attempted a
merge with the upstream, yielding a repo in a merging state, with a
huge list of dirtied files in the working tree, plus untracked files.

> > I personally would prefer portage simply adjusting its behavior based on the owner of the repository. That is, if it's the 'portage' user then assume full control, and if it is a different user, then fall back to a preserving, conservative mode of operation. Unfortunately, for me, this idea was received skeptically at best in a recent discussion in #gentoo-portage.
>
> This wouldn't work for Prefix and also FEATURES="-usersync".
>
> --
>
> Now, looking forward in a constructive manner, I think we can
> make both camps happy here with an option to indicate
> If Portage can assume the repository will be touched by something
> other than Portage.
>
> I propose a configuration option called 'volatile' (thanks
> Arsen for the name suggestion) which indicates that the
> repository may be changed outside of Portage by any time,
> and hence no destructive operations should be carried out.
>
> If the option is on, it also prohibits some optimisations
> which require assuming the integrity of the repository.
>
> I have a draft of these changes at https://github.com/gentoo/portage/pull/961.
>
> (https://github.com/gentoo/portage/pull/961.patch if want to view as a raw patch series.0
>
> I am undecided on whether volatile should be default on or not. In the PR
> as it is, it defaults to off, which would require a news item. I may just turn it
> on given the tone of this thread, as none of it has been very encouraging
> for further work on Portage.