Gentoo Logo
Gentoo Spaceship




Note: Due to technical difficulties, the Archives are currently not up to date. GMANE provides an alternative service for most mailing lists.
c.f. bug 424647
List Archive: gentoo-scm
Navigation:
Lists: gentoo-scm: < Prev By Thread Next > < Prev By Date Next >
Headers:
To: "Robin H. Johnson" <robbat2@g.o>
From: Donnie Berkholz <dberkholz@g.o>
Subject: Re: Status of the project?
Date: Mon, 26 Jan 2009 09:07:29 -0800
On 01:45 Sat 24 Jan     , Robin H. Johnson wrote:
> On Sat, Jan 24, 2009 at 01:39:43AM +0000, Mike Auty wrote:
> Secondary delay is that I asked for some input on hardware and tuning re
> managing a very large repo, and still haven't got a response from my
> contact (at kernel.org). Regardless of the tuning, we're going to need
> to figure out a box with lots of RAM for the repo size.

There was a conversation on #osuosl last week about gitweb setup for 
performance, which I attached. 

> > There seem to be two front runners, 1 repo/package and 1 repo/tree.  The
> > pros and cons are as set out in your file (the flat tree option pros
> > don't seem to outweigh the cons, so I haven't really considered that).
> 
> 1 repo/tree is the only one that will fly, see past discussion on the
> list.

I'm still not convinced of that. The alternative would require a patch 
to git-submodules, some additional tools support for grabbing additional 
repos with checkouts and repoman, but it has some compelling advantages 
(like non-devs don't need 1 GB of web space / bandwidth to post their 
forks of the tree).

On the other hand, it would be the easiest, and perhaps that's reason 
enough, because the people working on it don't have the time to do 
anything more.

-- 
Thanks,
Donnie

Donnie Berkholz
Developer, Gentoo Linux
Blog: http://dberkholz.wordpress.com
19:50 <   pquerna > hey, osuosl still do kernel.org stuff?
19:50 <   pquerna > i was wondering about gitweb setups
19:51 <    gchaix@> pquerna: yes, we do kernel.org stuff, but I'm not personally familiar with the gitweb setup
19:51 <   pquerna > mostly assuming they use mod_perl for it, as the other option of gitweb via CGI seems insane :)
19:51 <   pquerna > looking at doing more git stuff @apache
19:52 <    gchaix@> one of the systems guys will have to answer that (I'm just a developer at the OSL) :-)
19:53 -!- paul|laptop is now known as paul|tv
19:56 < dberkholz > gchaix: woohoo!
19:56 <    Jeff_S@> warthog9: ^^ is that question from pquerna something you could answer?
19:56 <  warthog9 > huh?
19:56 <  warthog9 > what was the question again?
19:57 <    Jeff_S@> gitweb setup...?
19:57 <  warthog9 > well gchaix you host kernel.org, I'm not sure you guys even have logins on the boxes though ;-)
19:58 <  warthog9 > pquerna- what you need to know about gitweb?
19:58 <    Jeff_S@> warthog9: thanks :)
19:59 <   pquerna > warthog9: we are looking at adding git to apache.org
19:59 <  warthog9 > pquerna- cool, how many git trees you have?
19:59 <   pquerna > warthog9: mostly, how is it setup to get reasonable performance , I assume you use mod_perl....
20:00 <   pquerna > warthog9: short term, few dozen, long term, hundreds.
20:00 <  warthog9 > bah mod_perl only helps a small fraction, no kernel.org runs two different forks of gitweb
20:00 <  warthog9 > there's gitweb-caching (mine) and gitweb-caching (lea's)
20:00 <  warthog9 > android.git.kernel.org is running lea's and git.kernel.org is running mine
20:01 <  warthog9 > slightly different approaches to caching but all told huge speed improvements
20:01 <   pquerna > no interest in upstream git of incorporating your changes?
20:02 <  warthog9 > pquerna- when I tried mine they refused, when Lea tried there was a lot of push back both from the "it's more complicated" route and she did a couple of things that some of the other dev's don't like
20:02 <   pquerna > hmm.
20:02 <   pquerna > so, if we wanted to run gitweb, I guess we would be best off pulling from one of yours.  I assume they are in a git tree somewhere :) ?
20:02 <  warthog9 > both could be cleaned up, but basically it seems to boil down if your running a big site, repo.or.cz, *kernel.org, etc your running a modded gitweb :-/
20:03 <  warthog9 > mine is on git.kernel.org, though it's a bit long in the tooth (I need to spend some time updating it and seeing if it's worth keeping)
20:03 <  warthog9 > Lea's is on repo.or.cz
20:03 <  warthog9 > mine is 'easier' to setup, but I trade performance for disk space, I.E. it's not uncommon for me to have 20G of disk space chewed up in the caching
20:04 <  warthog9 > Lea's uses memcached which apparently the distro's haven't all caught up with including by default so it can be a bit of a hoop jump
20:04 <   pquerna > is your git.kernel.org easily hosted on one machine?
20:05 <   pquerna > trying to figure out how much hw we need to toss at this
20:05 <  warthog9 > yeah, both lea's and mine are on single machines
20:05 <  warthog9 > (actually the same machine but yeah)
20:06 <   pquerna > so, just disk io / disk space I guess.
20:07 <  warthog9 > yeah, but reading a file is cheaper than all the git calls in terms of i/o
20:10 <   pquerna > are any of the other git web frontends... good?
20:11 <  warthog9 > there's the one in trunk, which works but is insanely i/o intensive particularly on larger setups and has the 'stampeding herd' problem
20:12 <  warthog9 > repo.or.cz is a much smaller patch to just deal with caching on the front page, and the author would argue it's a better approach than Lea's or mine, there's Lea's and mine and that's about it
20:12 <  warthog9 > well there's ummm what was it cgit?  it's a gitweb frontend written in C
20:13 <   pquerna > http://hjemli.net/git/cgit/
20:13 <   pquerna > yeah.
20:13 <   pquerna > hmm
20:13 <  warthog9 > it's fast and good, but the url's for it and the normal gitweb are incompatible, so while I would genuinely consider switching - I can't because of the url problem.  It also doesn't have all the features of gitweb which might annoy people
20:13 <   pquerna > http://cgit.freedesktop.org/
20:13 <   pquerna > right
Attachment:
pgpdeKfL0iqOG.pgp (PGP signature)
Replies:
Re: Status of the project?
-- Mike Auty
Re: Status of the project?
-- Donnie Berkholz
References:
Status of the project?
-- Mike Auty
Re: Status of the project?
-- Donnie Berkholz
Re: Status of the project?
-- Mike Auty
Re: Status of the project?
-- Robin H. Johnson
Navigation:
Lists: gentoo-scm: < Prev By Thread Next > < Prev By Date Next >
Previous by thread:
Re: Status of the project?
Next by thread:
Re: Status of the project?
Previous by date:
Re: Status of the project?
Next by date:
Re: Status of the project?


Updated Jun 17, 2009

Summary: Archive of the gentoo-scm mailing list.

Donate to support our development efforts.

Copyright 2001-2013 Gentoo Foundation, Inc. Questions, Comments? Contact us.