Re: [gentoo-soc] Online Image builder Proposal

[Fabian Groffen]

On 02-04-2009 20:32:21 +0000, ethankhall@×××××.com wrote:
> Online Image Builder
>
> Objective: To provide the ability for the Gentoo users to have
> installable images built by a web server that would contact them when
> it was ready to be downloaded.
>
> Abstract:  Using example front ends create a way to deliver a generic
> Gentoo system without having to have a user there to do the
> installation.  The delivery would then be provided in several formats
> (zip, tar.bz, bootable iso).  The user would be expected to install
> their own kernel, a genkernel will be shipped with the installation
> image but will not be suitable for every user so they *should* go and
> recompile it with their own options.

Just in case you don't have enough work.  The "prefix" files on
tinderbox.d.g.o do not come with a kernel or libc, as they are no
standalone systems.  Though you still can build an image out of them,
mounted/extracted to a given place (/var/tmp?).  They are even
unprivileged by default, so no root rights involved.

> Deliverable's:
>
>   • A website (in PHP or Python) with the ability to take orders from
>     users, que them, and notify users when their build is ready to be
>     downloaded.

Do you think of AJAX based strategies, or e.g. notification per e-mail
for busy times?  It's a waste if a user reloads or never picks up the
generated image that hold up others in your queue, of course.

>   • User documentation in an easy to understand format.
>   • A modular design so new things can be put into place with ease.
>   • A generic base system for x86 and amd64 (server and desktop of
>     each).
>
>
> Timeline:
> May 10th - 24th:
>
>   • Get a standard set of use flags that will be applied to every
>     package (will be different for each system).

USE-flags are not really within your control, IMO.  You can only use the
USE-flags as used in each individual binpkg.  The most you can do is to
suggest default USE-flags for the builders.  In that area I see some
opportunities to spend some time, since you'll have to trade-off
usefullness of a package against how many you're going to need (and
possible unwanted features/dependencies).  Nice example is PHP which
might not be really usefull with most flags turned off.

>   • Get a standard set of packages that will be installed on each
>     system.

@system perhaps?

>   • Decide on standard CFLAGS to be used, most likly CFLAGS="-O2 -march
>     =i686 -pipe"

You will *have* to use very modest flags since you target a generic
arch.  So your example is fine, I think catalyst/portage-make.conf comes
with some sane defaults for the other arches.

>   • Begin work on Python script to do the install.
>   • Decide what platform will do the building.

Something fast, probably x86_64 ;)

> May 24th - June 8th:
>
>   • Have Python script done and work on debugging it.
>   • Have the finalized set of packages that will be available to be
>     installed though the online tool.

Doesn't that depend on what is available to you (or what you can get
built)

>   • Begin work on the que system.
>   • Begin front end mock up/decide what mock up to use.
>
> June 8th - June 22nd:
>
>   • Do a build via the online (protected) front end and debug w/ que
>     utility.
>   • Decide what to include with each delivery.
>   • Prepare a bootable iso template.
>   • Have everything that is build saved in a binary form (to save time
>     for each build).
>
> June 22nd - 29th:
>
>   • Flex time for unanticipated problems.
>
> June 29th - July 20th:
>
>   • Have multiple people build systems concurrently(while website still
>     protected).  Test system tax's.
>   • Work out any bugs on the delivery system's.
>   • Find and fix system bottlenecks.
>
> July 20th - August 10th:
>
>   • Have the site open to the gentoo population with a request for
>     error's that happen on build.
>   • Fix error's.
>   • Generate a time from request to delivery time sample.
>   • Be able to predict (withing an hour or two) when a user's build
>     will be ready.
>   • Find and fix system bottlenecks.
>
> August 10th - Onward:
>
>   • Improve delivery system.
>   • Find and fix system bottlenecks.
>   • Fix error's.
>   • Make build system portable (so it can be taken to different
>     location ie. university, and have builds done by there servers with
>     even more customizability).

The coolest thing of course would be when you could select a predefined
image to be created for your arch, which then allows you to download a
preconfigured Gentoo installation that runs the image builder website :)


-- 
Fabian Groffen
Gentoo on a different level