Re: [gentoo-dev] The new Portage Job System

[Dan Armak]

On Saturday 11 August 2001 16:37, you wrote:
> Den 11 Aug 2001 12:05:01 +0300 skrev Dan Armak:
> > Hi all,
>
> Hi!
>
> As I was in the discussion on IRC #gentoo I agree that we should have
> something like this, I do howevery disagree on some of the stuff
> below...
>
> > Under profiles/jobs, or in a similar location, will be a job dir
> > (Possibly profiles/default/jobs, and the references may then be
> > redirected through /etc/make.profile). Developers can place jobs in it
> > for Portage to execute.
>
> Should it really be possible to redirect this? I guess it should always
> be in the same place in portage and should be redirected by redirecting
> the portage-tree.
No, you didn't understand. My use of "redirect" was ambigous. 
/etc/make.profile is a symlink to /usr/portage/profiles/deafult (check your 
system, it's there). So I said a user might redirect references to 
/etc/make.profile with the same result as if he used 
/usr/portage/profiles/default directly. It's not a very important point.


>
> > Helper files:
> > ----------------
> > Jobs can have misc files, a separate dir should be given to these i.e.
> > profiles/jobs/files.
>
> Agreed.
>
> > Generic scripts:
> > --------------------
> > Provide a dir profiles/jobs/scripts and populate it with *generic*
> > scripts. Under profiles/jobs put job spec files which, in their first few
> > (commented out) lines, have info about when to run them, and then have a
> > #! line that specifies one of these generic scripts as the interpreter.
> > For example, a job spec for moving a package would be called
> > move-koffice.job, and begin:
> >
> > # Job spec description: move the koffice package from kde-apps to
> > app-office # Run Once
> > # Author: Dan Armak <danarmak@g.o>
> > # $Header: $
> > #! /usr/portage/profiles/jobs/scripts/movepackage.sh
> > move koffice kde-apps app-office
> > ...
> > and so on, from here script-specific info would follow. This is the best
> > model IMHO.
> > Of course, under this model a job file can still be a script in its own
> > right, and use none of the generic scripts.
>
> I really don't like this approach. This way a wrong written script might
> corrupt a users database. I think it's better if we have a few scripts
> (i.e. those we have found out we need) and then we use the job-files for
> giving parameters to the script. Like in the case of moving a package we
> could have something like:
>
> In job-file:
>
> type: rellocate
> author: Mikael Hallendal <hallski@g.o>
> version: $Header$
> control-file: koffice-to-app-office.control
>
> the control-file is then in files/
>
> portage takes a look at type, recognizes it as a rellocate-type of job.
> It excecutes:
> '/usr/lib/portage/bin/rellocate
> /usr/portage/profiles/jobs/files/koffice-to-app-office.control'
>
> Only the system-team may add/approve scripts into portage so that we get
> some control of what goes in. This way a badly written job-file will
> result in an error when excecuting the script. A badly written
> control-file might do more damage but I think this is unavoidable and
> the control-file should have a really easy syntax.
That's just what I meant!! The "generic" scripts are what you want trhe 
system team to approve, I agree with that. Then, any developer may add what I 
called "job spec files", which you called control-files - that's just what I  
meant. We just said the same thing in different terms.

>
> > The Portage-handled header: (I don't like this but have no better
> > ideas)
>
> Hmm, I'm not sure I follow you here. What is this for?
I'm not sure I followed your comments either, I've tried to answer them:

>
> > When to run - options:
> > -----------------------------
> > 1. The jobs will be executed after each run of emerge, including emerge
> > rsync.
>
> What types of jobs would be needed to run after each emerge?
I don't knwo - 

>
> > 2. Before each run of emerge, to accommodate the following scenario:
> > change in cvs requires job to run before any emerges.
>
> How will this be of any use. If something changed in the cvs it will
> break the users tree at the same time he gets the job-descr. the actuall
> job wouldn't run until next rsync.
I don't follow, sorry.

>
> > 4. Provide "emerge jobs" command to run on demand.
>
> I think that we should have a 'emerge jobs'-kinda a tool which should be
> used by developers which don't use 'emerge rsync'. I do however think
> that jobs for users should be run by 'emerge rsync' and no otherwise.
No, I didn't mean "emerge jobs" to download new jobs. I meant "emergre jobs" 
to execute currently present jobs, or a specific job. For example, a job that 
runs a configure script or interface for some package, and you want to 
reconfigure it. You don't want to remerge it, so you run the job yourself.

>
> > Ideas/needs for jobs:
> > ---------------------------
> > 1. Move a package to a different category, update /var/db/pkg if the
> > package was already installed. This is what we're doing with kde/gnome.
>
> This is indeed a very much needed.
>
> > 2. Output some info to the user. For example about an important new
> > change in Portage/emerge/new KDE version etc., for the users who don't
> > read the cvs logs or gentoo-dev.
>
> This is also a good feature. But should only be run after an emerge
> rsync.
A job can specify when it should be run.

>
> > 3. Merge a new version of sys-apps/portage when this is needed for some
> > ebuilds' new versions to work, like in the \" story. A user who got the
> > new ebuilds with non-escaped quotes but didn't update portage would be in
> > big trouble.
>
> Perhaps this should be handled either by (2), ie. give the user a
> message about upgrading there portage-system. Or by actually doing
> inside of 'emerge rsync'. The problem with automatically do stuff is
> that there are people (me for one) that don't like when things happend
> by themself. I like to have control of when I do update a package.
You can change the automatic/manual control of these things of course. A user 
who only uses Portage to emerge new apps he needs doesn't need to keep track 
of updates to Portage itself, just emerge rsync once in a while.

>
> > 4. After merging a package, output usage intructions. Deprecate the
> > pkg_postinst usage, emerge outputs info after it anyway.
> >
> > 5. As above, run a configuration program.
> >
> > 6. Manage a bug report system. A user who got an error running one of uor
> > ebuilds would get this job executed.
> > ...
>
> 4-5 should be handled inside the ebuild (imho). I see no reason why it
> should be in the jobs-system instead. If a package needs to be
> configured afterwords it should be done in pkg_postinst (). Same for
> giving the user a notice about the package.
Well, pkg_postinst comes before annoying portage messages that move it off 
the screen. Also currently ebuilds are supposed to be always non-interactive, 
not even e.g. displaying a help message with most but only with echo. Jobs 
can be interactive.

>
> 6: would be a nice feature, but I don't think that it should be a job,
> it should be in portage system itself. The system should give a question
> if it fails to merge an ebuild asking if the user would like to report
> the bug. If so it starts the routines for doing so and puts the results
> in /var/db/bug-reports or something, then when doing emerge rsync
> portage looks in /var/db/bug-reports and sends them somewhere.

I was only making suggestions, trying to think up ideas. I'm not saying all 
these things should be made jobs, or only them.




-- 

Dan Armak
Gentoo Linux Developer, Desktop Team
Matan, Israel