| 1 |
Alan McKinnon <alan.mckinnon <at> gmail.com> writes: |
| 2 |
|
| 3 |
|
| 4 |
> > often a custom, scheduler/framework has to be modified or custom developed, |
| 5 |
> > but I find it hard to believe there is no way to massively speed up |
| 6 |
> > something like gitlab on a robust linux cluster. That dragon has been |
| 7 |
> > sleighed for most all load problems, be it HPC, networking issues, or |
| 8 |
> > security audits, etc etc. Often, a cloud/cluster software can be radically |
| 9 |
> > sped up by allocating lots of extra ram to the framework it is running on. |
| 10 |
> > Cloud vendors charge way to much for extra ram, so performance of |
| 11 |
> > ram-intensive services are often run where there is ample ram. |
| 12 |
|
| 13 |
> Depends. It might all be running on a single-threaded ruby process in |
| 14 |
> the middleware |
| 15 |
|
| 16 |
|
| 17 |
If you believe what one reads:: |
| 18 |
|
| 19 |
"You seem to be confusing two very different things here: the Ruby |
| 20 |
Programming Language and the specific threading model of one specific |
| 21 |
implementation of the Ruby Programming Language. There are currently around |
| 22 |
11 different implementations of the Ruby Programming Language, with very |
| 23 |
different and unique threading models. " circa 2011. |
| 24 |
|
| 25 |
> Don't laugh, I've seen stranger things. |
| 26 |
> Alan |
| 27 |
|
| 28 |
There are a multitude of ways to profile codes and the modules. Surely |
| 29 |
someone at gitlab has done this? (Evidently No?) A big part of cluster |
| 30 |
architecture is to identify the bottlenecks in codes that run on a single |
| 31 |
machine and implement solutions to lessen the bottlenecks. Most modern |
| 32 |
codes inherently run parallel and that's what cluster provide, more |
| 33 |
resources for many more threads, and the addition of custom hardware, such |
| 34 |
as GPUs, DDR5 ram, fpga and different architectures of processors. Surely |
| 35 |
gitlab can be fixed with a correctly applied cluster, or the codes that |
| 36 |
compose gitlab, need to be fixed... |
| 37 |
|
| 38 |
I'll be the first one to verify, you just don't rent a cluster |
| 39 |
(containers/VM) and move you slow applications there and magically |
| 40 |
all is fixed. The first step is to identify and qualify/quantify |
| 41 |
the source(s) of the bottlnecks. CS-101. |
| 42 |
|
| 43 |
However, profiling code/kernels/libs often leads one down the 'fuzzer' path, |
| 44 |
and that's a whole other (fun) time-sink. |
| 45 |
|
| 46 |
ymmv, |
| 47 |
James |
Archives