List Archive: gentoo-scm
Note: Due to technical difficulties, the Archives are currently not up to date.
provides an alternative service for most mailing lists.c.f. bug 424647
On 01:45 Sat 24 Jan , Robin H. Johnson wrote:
> On Sat, Jan 24, 2009 at 01:39:43AM +0000, Mike Auty wrote:
> Secondary delay is that I asked for some input on hardware and tuning re
> managing a very large repo, and still haven't got a response from my
> contact (at kernel.org). Regardless of the tuning, we're going to need
> to figure out a box with lots of RAM for the repo size.
There was a conversation on #osuosl last week about gitweb setup for
performance, which I attached.
> > There seem to be two front runners, 1 repo/package and 1 repo/tree. The
> > pros and cons are as set out in your file (the flat tree option pros
> > don't seem to outweigh the cons, so I haven't really considered that).
> 1 repo/tree is the only one that will fly, see past discussion on the
I'm still not convinced of that. The alternative would require a patch
to git-submodules, some additional tools support for grabbing additional
repos with checkouts and repoman, but it has some compelling advantages
(like non-devs don't need 1 GB of web space / bandwidth to post their
forks of the tree).
On the other hand, it would be the easiest, and perhaps that's reason
enough, because the people working on it don't have the time to do
Developer, Gentoo Linux
19:50 < pquerna > hey, osuosl still do kernel.org stuff?
19:50 < pquerna > i was wondering about gitweb setups
19:51 < gchaix@> pquerna: yes, we do kernel.org stuff, but I'm not personally familiar with the gitweb setup
19:51 < pquerna > mostly assuming they use mod_perl for it, as the other option of gitweb via CGI seems insane :)
19:51 < pquerna > looking at doing more git stuff @apache
19:52 < gchaix@> one of the systems guys will have to answer that (I'm just a developer at the OSL) :-)
19:53 -!- paul|laptop is now known as paul|tv
19:56 < dberkholz > gchaix: woohoo!
19:56 < Jeff_S@> warthog9: ^^ is that question from pquerna something you could answer?
19:56 < warthog9 > huh?
19:56 < warthog9 > what was the question again?
19:57 < Jeff_S@> gitweb setup...?
19:57 < warthog9 > well gchaix you host kernel.org, I'm not sure you guys even have logins on the boxes though ;-)
19:58 < warthog9 > pquerna- what you need to know about gitweb?
19:58 < Jeff_S@> warthog9: thanks :)
19:59 < pquerna > warthog9: we are looking at adding git to apache.org
19:59 < warthog9 > pquerna- cool, how many git trees you have?
19:59 < pquerna > warthog9: mostly, how is it setup to get reasonable performance , I assume you use mod_perl....
20:00 < pquerna > warthog9: short term, few dozen, long term, hundreds.
20:00 < warthog9 > bah mod_perl only helps a small fraction, no kernel.org runs two different forks of gitweb
20:00 < warthog9 > there's gitweb-caching (mine) and gitweb-caching (lea's)
20:00 < warthog9 > android.git.kernel.org is running lea's and git.kernel.org is running mine
20:01 < warthog9 > slightly different approaches to caching but all told huge speed improvements
20:01 < pquerna > no interest in upstream git of incorporating your changes?
20:02 < warthog9 > pquerna- when I tried mine they refused, when Lea tried there was a lot of push back both from the "it's more complicated" route and she did a couple of things that some of the other dev's don't like
20:02 < pquerna > hmm.
20:02 < pquerna > so, if we wanted to run gitweb, I guess we would be best off pulling from one of yours. I assume they are in a git tree somewhere :) ?
20:02 < warthog9 > both could be cleaned up, but basically it seems to boil down if your running a big site, repo.or.cz, *kernel.org, etc your running a modded gitweb :-/
20:03 < warthog9 > mine is on git.kernel.org, though it's a bit long in the tooth (I need to spend some time updating it and seeing if it's worth keeping)
20:03 < warthog9 > Lea's is on repo.or.cz
20:03 < warthog9 > mine is 'easier' to setup, but I trade performance for disk space, I.E. it's not uncommon for me to have 20G of disk space chewed up in the caching
20:04 < warthog9 > Lea's uses memcached which apparently the distro's haven't all caught up with including by default so it can be a bit of a hoop jump
20:04 < pquerna > is your git.kernel.org easily hosted on one machine?
20:05 < pquerna > trying to figure out how much hw we need to toss at this
20:05 < warthog9 > yeah, both lea's and mine are on single machines
20:05 < warthog9 > (actually the same machine but yeah)
20:06 < pquerna > so, just disk io / disk space I guess.
20:07 < warthog9 > yeah, but reading a file is cheaper than all the git calls in terms of i/o
20:10 < pquerna > are any of the other git web frontends... good?
20:11 < warthog9 > there's the one in trunk, which works but is insanely i/o intensive particularly on larger setups and has the 'stampeding herd' problem
20:12 < warthog9 > repo.or.cz is a much smaller patch to just deal with caching on the front page, and the author would argue it's a better approach than Lea's or mine, there's Lea's and mine and that's about it
20:12 < warthog9 > well there's ummm what was it cgit? it's a gitweb frontend written in C
20:13 < pquerna > http://hjemli.net/git/cgit/
20:13 < pquerna > yeah.
20:13 < pquerna > hmm
20:13 < warthog9 > it's fast and good, but the url's for it and the normal gitweb are incompatible, so while I would genuinely consider switching - I can't because of the url problem. It also doesn't have all the features of gitweb which might annoy people
20:13 < pquerna > http://cgit.freedesktop.org/
20:13 < pquerna > right
Updated Jun 17, 2009
Archive of the gentoo-scm mailing list.
Donate to support our development efforts.