[gentoo-user] Re: Gitlab experiences

[James]

Jeremi Piotrowski <jeremi.piotrowski <at> gmail.com> writes:


> > > Has anyone attempted to install a self hosted gitlab  on gentoo server(s)?


> I would deploy it with docker. The gitlab guys push official images of the
> main gitlab app[1] and CI runners[2] to dockerhub. That should be
> the easiest path to getting it up and running in no time.

Docker runs everywhere, including Mesos clusters.

> That being said, gitlab does not really play well with clustering in
> general. I don't think the main part of the app does any kind of
> horizontal scaling (gitlab.com is hosted on a single server) so you need a
> fairly beefy server.

That's weird. Almost every type of heavy load is finding it's way to
clusters now; many via containers some on bare metal clusters. Granted, very
often a custom, scheduler/framework has to be modified or custom developed,
but I find it hard to believe there is no way to massively speed up
something like gitlab  on a robust linux cluster. That dragon has been
sleighed for most all load problems, be it HPC, networking issues, or
security audits, etc etc. Often, a cloud/cluster software can be radically
sped up by allocating lots of extra ram to the framework it is running on.
Cloud vendors charge way to much for extra ram, so performance of
ram-intensive services are often run where there is ample ram.

> And while storage should be entirely up to you (the
> app _should_ be indifferent to what you use) most folks appear to run with
> local disks or NFS.

NFS is yesterday's cruft. Many are now using cephfs to replace NFS. Surely a
distributed file system (DFS) to work with the (master/slave) nodes in a
cluster is also keenly important. Granted the daddy of all cluster file
systems (HDFS) is a pig, but it is being actively replaces with a myriad of
DFS solutions. Using NFS in a linux cluster turns performance into a
cluster*F!!!. Just say no to NFS and learn about DFS. NFS is what clusters
used 15 years ago (beowulf/pvm). OrangeFS is an outgrowth if the old PVM
and is now part of the kernel (4.6+).

Any and all references as to why gitlab crawls, the more technical the
better, are of keen interest to me. It's not that I do not believe what you
are saying, it just more likely that someone with keen cluster skills has
not looked at gitlab. ymmv. The fact of what you are saying that gitlab does
not run fast on a cluster, is very bewildering to me, and wreaks of poor
architecture on the cluster it was tested on, more than anything else.
Scheduler/config/frameworks are properly needed to get large loads to fly on
a cluster. At this point, it a lot of work and a bit of black-arts....
Folks with deep pockets, are not failing to make any workload fly, in my
extensive research experiences.


> > > Any experiences with gitlab are most welcome for comment, good or bad.
> > Yes. Bad. Slow, unreactive, eats tons of resources. Doesn't scale with
> > large repos (except you have unlimited access to hardware resources). A
> > Linux kernel git mirror finally crashed it.

What are the specifics of the  mirror setup, DFS and the crash?

> I can second that - it's slow and load times are long even when performing
> the most basic operations such as opening a small file or viewing a single
> commit (you _will_ notice - it regulary takes several seconds). Mind you
> the gitlab folks are working on improving this since several releases.

Sounds like they need a cluster architect to help them select the relevant
cluster tools and tune/modify/develop a custom scheduler and frameworks, for
each of the sub problems. Also, most projects are fairly clueless about
tuning the underlying linux kernels for cluster operations. Stock vendor
kernels are 'dog-crap' for linux clusters, but that is another entire set of
problems.


> As a user, another issue I have with it is that the merge request/review
> interface is just terrible. There is _no_ merge request versioning, so
> either you submit your code in perfect shape at first try, or any change
> (amending/rebasing/merging) will cause the changelist to be duplicated
> many times over. You also lose track of the review history instantly - old
> comments are either concealed or swallowed.

> There is also no CLI utility to automate common review-related tasks
> (submitting/responding to review) so you are forced to do everything over
> the slow WebUI.

That does sound bad...

> If you care at all about the codereview aspects, I would recommend gerrit
> or phabricator. Both have cli utilities (git-review and arcanist) and
> while some claim they are ugly (heard that one especially about
> gerrit) they are 100x more practical.

> If you only care about having a repository browser then gitlab can work
> but there are simpler apps out there (gogs/pagure).

Interesting comments.

thanks,
James