Re: [gentoo-dev] Deprecating repoman

[Joshua Kinard]

On 3/10/2022 14:58, Alec Warner wrote:
> On Thu, Mar 10, 2022 at 10:27 AM Joshua Kinard <kumba@g.o> wrote:
>>
>> On 3/9/2022 16:00, Matt Turner wrote:
>>> I'd like to deprecate and ultimately remove repoman. I believe that
>>> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
>>> are far superior replacements, and it makes sense to have people using
>>> the same tool and seeing the same warnings as in the CI.
>>>
>>> Are there any useful checks or behaviors of repoman that are missing
>>> from pkgcheck and pkgcommit?
>>>
>>> Thanks,
>>> Matt
>>
>> repoman has been //the// goto tool for checking in a change since before I
>> was a developer in 2003.  It does everything we need, in one simple tool.
>> Your proposal looks to replace repoman's functionality (and snark) with two
>> or more packages, including tools from a package named after a fellow developer.
>>
>> Additionally, "I believe that <foo> are far superior replacements" is an
>> entirely subjective opinion and, frankly, completely invalid as
>> justification to replace a tool that has worked fine for the last 20+ years.
>>  What is so fundamentally broken about repoman that cannot be fixed such
>> that it needs total replacement by multiple independent tools?  Please
>> document. the pros and cons here so that we can all make an informed decision.
>
> So here is the more basic argument, you can agree or disagree.
>
> *The goal I want is for people to commit to the tree and not break things.*

This goal I agree with, and I am rather pedantic about.  If not mildly
fearful of.  We've all been there at least once in our dev-lives.  It's
almost a rite of passage, if you will, to break the tree with a dumb commit
at least once.  Goal is to never repeat that mistake again.


> If we could accomplish this with no tooling at all, that would be
> great. Sadly humans are fallible and so we have tools to check their
> work. Currently this tooling has two parts:
>  - pre-push tooling that you run prior to pushing.
>  - post-push CI tooling that informs you when you break the tree.
>
> So holding to our goal of not breaking the tree, it's better for
> developers to run the same QA tool in pre-push that CI is using,
> because our metric for 'breaking the tree' is the CI tool and if the
> CI tool passes locally, there is a strong likelihood it will not break
> in CI either. Note this argument is generic, I'm not even saying which
> tools are in use, or which tools are better, or anything like that.
>
> Here we see Matt discussing a workflow he finds frustrating.
>  - A developer does a push.
>  - Their push breaks CI.
>  - He inquires about their workflow.
>  - He learns they did not run the CI tool in their pre-push workflow.
>  - He tells them they should run the CI tool in their pre-push workflow.
>  - This happens many times, causing this thread.
>
> The point of the thread then is to convince people to run the CI tool
> in pre-push, as a matter of policy, to reduce tree breakage and reduce
> the occurrence of the above conversation.

I stick to the officially-published method of checking and committing changes:
https://devmanual.gentoo.org/ebuild-maintenance/git/index.html

The two tools highlighted there for the bulk of the work is repoman and
pkgdev.  repoman is cited twelve times, pkgdev is cited six times.  pkgcheck
is mentioned once.  pkgcommit has no mentions.

From that, one should not be faulted for assuming that repoman is the more
important tool, if not preferred tool, with pkgdev coming in second place.
pkgcheck comes across as entirely optional and even seems equivalent to
'repoman full', and how would one know that pkgcommit even exists?

Not all of us devs are involved in the automated CI stuff that goes on
behind the scenes.  Matt and Michał are, so they know how bad it can be and
the likely countless hours spent fixing things or yelling at others when
these mistakes are made.  But if this isn't documented somewhere, how are
those of us not involved in the CI-side able to keep up?  When I am in doubt
over something, I go to the documentation to address things.  Right now,
that documentation is going to more or less tell me to stick with repoman.

That all said, most of my checks are done pre-commit, not pre-push.  It's
annoying trying to unwind a commit with a mistake in git, so I aim to not
have any in the commit itself.  By the time I get to the push stage,
everything MUST be perfect before I hit 'git push'.  Cause once something is
public, there's no going back.  In a handful of cases, I've gone as far as
nuking my local copy of the tree, re-cloning it, and re-doing everything
again just to fix a mistake that I should have caught pre-commit.


> So the generic argument above, we now get into the specifics.
>
>>
>> I'm not opposed to making our tools better, but I think before anything can
>> replace the RepoMan, several more boxes need to be ticked:
>>
>> - functionality from multiple tools should be packaged into a single tool.
>>   * caveat: at least provide a wrapper that, using args, can invoke the
>>     individual tools if needed.
>
> I do not believe it's reasonable to provide a 'drop-in' backwards
> compatibility tool guarantee.
> I believe we should provide a table of common repoman actions, with a
> mapping to their replacements; so that users can understand how to do
> similar actions.

Not opposed to this.  It's a reasonable alternative to crafting a wrapper
script or kludging in deprecated argument parsing to achieve likely limited
backwards compatibility.


>> - app-portage/mgorny-dev-scripts needs a new name.  It's fine if it's
>> intended to only be a collection of useful scripts for individual developers
>> on an as-needed basis, but if it's to be the Official Tool(TM), then it
>> needs to have a proper name.  If not all of the scripts contained within it
>> are applicable to the sole function of checking a change into the tree, then
>> only the scripts that deal with change validation and committing should be
>> broken out into a separate package, and ostensibly combined with any other
>> tools/scripts into a single package, and preferably a single tool, to get
>> the job done.
>
> I don't consider this a blocker, but I think it's mostly a discussion
> between mattst88 and mgorny.

My fault that I should have stated that I don't think it should block, but I
do think this it looks....hacky?  I mean, say a developer from another
distro gets curious about how Gentoo developers do their magic, and while
reading the devmanual, sees that the officially-sanctioned way to do
something developer-related is to install
app-portage/kumbas-super-awesome-devscripts.  It looks wildly out of place
in official documentation.  We used to have a package called
app-portage/gentoolkit-dev that contained a collection of utilities intended
for Gentoo developers.  I'd argue that maybe something like that would be a
good starting point for a new name.


>>
>> - all of our developer documentation needs to be updated to reference the
>> new tool and the new way of doing things, as well as a warning not to use
>> repoman any further after a set date.  Additionally, a news item is probably
>> advisable so that developers and proxy maintainers alike can get the message.
>
> We should hold Matt accountable for updating any relevant development
> documentation, including the table I mentioned above.
> I'm not sure a news item is strictly necessary, we might just p.mask
> repoman with a link to the guide that Matt will need to write about
> how repoman is being replaced.

How about making an edit to repoman to have it throw a deprecation warning
for itself and to install whatever we all agree should replace it?


>> - we need the snark.  Gentoo is all about personality, and RepoMan has been
>> scouring our neighborhoods for two decades and change, and while some may
>> think this is utterly frivolous, I will actually miss seeing those messages
>> on the console every time I commit a change.  They give the RepoMan
>> personality and a soul, and thus, contribute to the uniqueness that is Gentoo.
>
> Cool.

Back in mid-2002, it was exactly Gentoo's snarkness that tipped the scales
for me.  I'd just given up on FreeBSD-4.x on a sparc64 machine (old Sun
Blade 100) and LFS turned out to not be a lot of fun, but Gentoo worked fine
on it.  All of the colors on the terminal gave it zing and pop, and made it
rather fun to work with.  rpm and apt-get were dull;  emerge was cheeky and
playful!  Still is to this day.

That sparc64 machine eventually became my very first dev machine when
Seemant was looking for someone to help out with the MIPS port.  I got a
laugh when I ran repoman for the first time and it tells me it is stalking
my neighborhood, looking for issues.  The developer tools were just as
cheeky and playful as the user-facing tools!

And we're still the only distro with some rad mascots, like Larry the Cow
and Znurt the alien (at least I think it's an alien -- bottom of the main
page, right of the Questions/Comments).  And whatever the thing at the very
bottom-right looking up is.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@g.o
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic