Gentoo Archives: gentoo-scm

From: Maciej Mrozowski <reavertm@×××××.com>
To: kde-scm-interest@×××.org
Cc: gentoo-scm@l.g.o
Subject: [gentoo-scm] Re: [Kde-scm-interest] Package splitting
Date: Thu, 28 Jan 2010 02:05:42
On Thursday 28 of January 2010 00:58:59 Oswald Buddenhagen wrote:
> On Wed, Jan 27, 2010 at 10:46:52PM +0100, Thiago Macieira wrote: > > [...] The build rules are: [...] > > why not (mostly) keep our current layout in form of a meta structure (be > it git submodules or not)? build order would be determined by cmakelists > which declare subdirectories (and *nothing* else) as ever. breaking out > of that layout would happen at own peril, but any breakage will be > quickly detected anyway (as the projects' cmakelists must be > self-contained and dependency-complete). > > On Wed, Jan 27, 2010 at 10:56:21PM +0100, Thiago Macieira wrote: > > And here's a suggestion to address those points, so a modification to the > > rules in the earlier email. > > > > Instead of having 4 stages per category, we actually have: > > libs > > libs2 > > libs3 > > ... > > you know, sysvinit is now moving away from fixed priorities to a > dependency-based system ... ;) > anyway, my private build script does effectively the same, so it isn't > that bad as such. :D
I'll drop in some comments here as in Gentoo we run into similar problems when trying to migrate to git (we did not do it yet, details follows). Mentioned atomicity This is actually what (I think - unfortunately) caused that our package repository is not going to be split into subprojects (like KDE, GNOME, WebApps, Java, and such). Our repository is in following format: category1/package1/package1-version.ebuild (along with some other files) category2/package2/package2-version.ebuild ... profiles/<subprofile> (along with many other files) (skip to "Git dependencies" if you don't care) There are inter-dependencies (also mutual - like for instance blocks) between packages, and what's the most important - profiles/ subdirectory contains repository-wide settings, so the simplest approach is to provide atomicity by keeping all in one repository. It's all fine with this, but one problem - purely technical - repository is huge enough (not as big as KDE one though) to make cloning it directly impractical as repacking on server side will simply kill the server (thus clones has to be delivered in alternative way - like git bundles or via rsync - and direct cloning explicitly disabled on server by the means of patching git). This is what actually holds our migration as splitting repository would need extensive changes to our package manager - repository dependencies, making profiles/ local to repository, and creating for instance master repository holding only profiles/ - global settings - and adding it as dependency for any other. Git "dependencies" This is quite similar to what you're trying to achieve with KDE - as you need to have some means to 'fake' git repository dependencies - either by the meaning "exactly this version of kdelibs.git" or more likely - "version of kdelibs.git at least". To be frank I don't exactly understand how would you achieve it automatically - manually - it would need to create some metadata file in toplevel directory of particular git repository for instance with following format (here as bash array but that's just example - file metadata toplevel repo dir): need=( <path_to_repo>:<branch>[:<commit>] ... ) Where <path_to_repo> would be relative path to particular repository - for instance 'kdelibs.git' or 'kdegames/libkdegames.git', <branch> being the name of the branch ("4.4", "master" etc) and optional <commit> would be any name that git understands as a commit identifier - so tag, sha1. For correctness branch 'tip' names (like 'master') should probably be forbidden. I really don't think there's need to invent new layout - your module layout on server is quite nice already (maybe just add 'playground' and review 'subdirectories' there like Thiago proposed_, so (layout on server) : cmakemodules.git (read below) kdelibs.git kdegames/ libkdegames.git kpat.git review/ somnewshinygame.git playground/ somebleedingedgegame.git Of course git itself doesn't understand any dependencies so you'd need to provide some scripts to clone repository AND recursively its dependencies (along with local hooks maybe to update those dependencies to match requirements in metadata - also this metadata would need to be updated manually when needed (for all remote branches independently). CMake modules I'd propose separate repository for them and adding it as a dependency of all other repositories that need it (also with version specifier) - it would prevent .cmake modules duplication. Update/clone script would need to clone this cmake modules repo to known local location (actually locations of all local clones need to be well defined if one is to include_directories from them for instance), so that all other projects (repositories) would just add in first line of CMakeLists.txt: set (CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../cmakemodules/") All this still need writing script to recursively create tags, tarballs (there's still possibility to package it the way it is now - as modules), recursively update dependencies (for instance working) or just update dependencies. As for work flow I think Thiago is on the spot that it's much safer to work on stable branch and forward-port bugfixes instead. Personally - there was very few of them - but it happened that I needed to reopen KDE bug marked as RESOLVED/FIXED just after fix has been commited to stable branch. My .5 -- regards MM