[gentoo-user] Re: Jenkins

[James]

Alec Ten Harmsel <alec <at> alectenharmsel.com> writes:


> On Sat, Jun 20, 2015 at 07:03:18PM +0000, James wrote:
> > Hello one and all,

> > I want to first install Jenkins on a single multicore amd system, so
> > I found this brief guide (which seems simple enough):
 
> > https://code.google.com/p/godin-gentoo-repository/wiki/Jenkins

> I highly recommend playing with the Jenkins docker container to get a
> feel for using the web UI:

>     docker run -p 8080:8080 -d jenkins

> The official jenkins image comes with the bare minimum of libraries so
> good luck building any C/C++ project, but it is helpful just to point
> and click around (if you already have docker, that is).

Yes, I've used Docker on gentoo before. NO, it's currently un-installed
and using docker for what I am after only complicates things at this point.


> This is not really a fair comparison. Jenkins has no concept of cores,
> only "executors". Each executor can run one build at a time. Jenkins was
> written with Java in mind (AFAIK), and Java build tools are the reason
> for this. `javac` by default launches some number of threads (not sure
> how many on other systems, but on my laptop it's usually 3) and compiles
> in parallel. I'm not sure it's possible to control this (someone feel
> free to correct me if I'm wrong), hence Jenkins does not care about
> cores and only about how many builds are allowed to run in parallel at
> once.

I'll have to drill into javac a bit more, it seems.

There is jenkins-bin on gentoo; it should (?) be a quick install and config.
I'll keep in mind what you are saying, but my main goal is to find (CI)
codes that I can run on gentoo-mesos clusters. Nothing about Jenkins is
a critical need for me. Although, as a consultant I do routinely get
asked about Jenkins experience, so just noodling around with it a bit
is a good idea for me.


> So, to the point; when you add more build slaves with more executors,
> you will be able to run more builds in parallel and it will therefore be
> faster. This gets even more complicated since you can constrain builds
> to only run on certain hosts and various other complicated setups, but
> in general more executors means more builds in parallel means faster.

AT some point, on a single multiprocessor system, Jenkins will slow down
even it more resources are configured for it (overall system load, if
nothing else)? AT that point, I can note the resources and apply those
limits to the cluster and note the performance differences. Surely, a
cluster with more resources will be able to do more (CI_Jenkins) work; so
discovering those details is a primary goal of this endeavour.



> My main suggestion is to not bother running a Jenkins cluster and stick
> with a single host setup unless:

Sorry, the cluster, is the main_goal, not Jenkins or CI. Buds at cisco
are all very braggadocios about CI (Jenkins and others) on their mesos
clusters.....

> * You are building so many things that utilization is near 100%
> * You are targeting multiple platforms and therefore need multiple build
>   hosts

YES, that is exactly the ultimate goal. A CI cluster, open to friends, pals
and customers (maybe). It's all about the mesos cluster, you should know that.

> P.S. I could not find any reference to parallel, threads, processes or
> anything else on javac's man page that makes me think it is easy to set
> the number of threads it uses.


No surprise there. Configuring threads, cgroups, and memory resources
is a hotly contested area of the cluster codes I have (am) working with.
Forget bikeshedding, it Blood_shedding....... especially now that RDMA
is hotly being pursued by the many.

Thanks for your insights,

James