[gentoo-dev] Re: Re: Versioning the tree

[Steve Long]

Chris Gianelloni wrote:

> I have a script which already does several things:
>
> #1. grabs "best_visible" for stable on each arch
> #2. repeat for each SLOT
> #3. purge unnecessary files from FILESDIR
> #4. strip to only "stable" profiles from profiles.desc
> #5. purge unnecessary USE from use.local.desc and use.desc
> #6. strip unused eclasses
> #7. regenerate Manifest (via ebuild $foo digest)
>
> I had also planned on it doing the following, but they haven't been
> implemented just yet:
>
> - strip all USE flags that aren't used from use.mask (per-profile)
> - strip all packages that aren't available from package.mask
> (per-profile)
>
> What this gives us is a *much* smaller tree, as it only includes stable
> packages/versions.
>
The smaller tree sounds great in terms of maintenance for the volunteers who
maintain the release.
 
>> Obviously maintaining that list is some work; predominantly watching
>> the announcements from the security team and fixing up dependencies,
>> and masking out new packages (for what that's worth).  It could be
>> regenerated on some or all releases, or perhaps on a yearly basis.  It
>> would also mean that versions listed there cannot be removed from the
>> tree (until they're bumped in the list).
>
> Well, the simpler approach was to simply copy over the newer, secure
> ebuilds, and any required dependencies, into the release tree.  There'd
> be no need to update mask files and such this way.  It would also be
> generated with the releases, automatically.
>
<snip> .. The one
> disadvantage to my design is it needs infra.  It needs it's own
> repository and rsync.
>
What does that entail? Would a co-located server suffice? 

(If it gets popular, I'd imagine those mirroring current rsyncs etc would
want to mirror the releases as well.)

> Basically, it makes the system act a bit more like some other
> development models.  We end up with the following:
>
> rsync://rsync.gentoo.org/gentoo-portage (this is equivalent to -current)
> rsync://rsync.gentoo.org/2007.0-release (this is the release tree)
>
> Now, the release trees are non-moving.  The 2007.0-release tree is
> *always* the 2007.0-release tree for as long as we decide to support it.
> Likely, this support period would begin as a single release and get
> extended as volunteers came around to support it.  New releases get
> their own tree.
>
Well count me in as a volunteer to help set this up and maintain an x86
release. I'm a pretty good coder if that helps.

> I also started writing a tool to "upgrade" the distribution.  The tool
> reads pre and post scripts for the upgrade, and performs the necessary
> steps.  Basically, a user can "dist-upgrade 2007.1" and the system
> would:
>
> - switch to the "2007.1" tree and "emerge --sync"
> - perform all "pre" steps from 2007.1
> - update world + revdep-rebuild + whatever else is necessary
> - perform all "post" steps
>
> With this, there would be a special "version" called "current" which
> would put the user on the "gentoo-portage" tree, AKA the tree we all
> know and love/loathe.  Release trees wouldn't have "arch" and "~arch" as
> everything would be stable there.  Testing would be done in the main
> tree.  This, in essence, gives us "testing", "release candidate" and
> "stable" environments, with the release trees being stable, and the
> current tree becoming test/release candidate.  Anything marked stable in
> the current tree would be a candidate for stable in the next release
> tree.
>
This sounds like a much more professional proposition to put to an IT exec.

> Users who want to use the current portage tree get what they want.
> Users who want a more stable tree get what they want.  Basically,
> everyone (hopefully) is happy, or at least as happy as we can feasibly
> make them.
>
This all sounds great. Respect for the work you've already put in.

other post:
> With the release trees, essentially only those
> interested in supporting the tree are required to work on it.  The tree
> is created entirely by scripts and is tested before it's released on
> the public.

Having it automated is definitely what I wanted to know about in the
original discussion.

>From your post we need to add:
> - strip all USE flags that aren't used from use.mask (per-profile)
> - strip all packages that aren't available from package.mask
> (per-profile)

What language is the script implemented in?

Wrt security updates, is it possible to tie into GLSAs so that we could
automate updating packages that need it? By updating I mean adding the
ebuilds and any dependencies (or dependants that might require updating.)

Caleb Cushing wrote:
> .. isn't it possible only to sync certain
> parts of the tree using excludes. maybe some additional functionality
> saying only sync package X for updates.

Would that help in terms of having say, up to date GUI packages, or is it
just easier to use overlays?

Chris Gianelloni wrote:

> No version changes on any packages, except those which are necessary due
> to a security violation, or a vulnerable package's dependencies.
>
I could imagine a situation where a dependant package (ie one using the
package updated) would also require an update. It'd be rare though, so I
guess it wouldn't need automating.

> Now, I don't know if it is the best approach, but it is the best one
> that I could come up with on my own that allows for a slow-moving tree
> with higher QA done on it (as the release trees are what we use for
> releases, they undergo an enormous amount of testing, in the default
> configuration, of course) and minimal changes.
>
It sounds like the right approach to me. There's all the custom approach of
gentoo, eg overlays, so users can always stay up to date with anything they
particularly want.

> Something that would be useful would be for package maintainers that
> wish to participate to perform further testing and validation on
> packages in the release snapshot prior to its final freeze.  This would
> give us even more eyes on the tree and hopefully provide a much higher
> quality snapshot once we're done.

It sounds like it'd also make the releases in general have better QA. What
do package/ herd maintainers think?

-- 
gentoo-dev@g.o mailing list