Re: [gentoo-dev] Re: mbox -- looks sort of interesting

[Александр Берсенев]

Hello,

I've updated the autodep and testing it now.

I've fixed a problem with the integration of autodep in the newest version
of Portage, but I think I did it in a hackish way. I want to ask if it is
possble to solve it better.

First, some words about the architecture of autodep. Autodep consists of
two parts:
1) A library, injected with LD_PRELOAD, which intercepting glibc functions
like open, read, write, etc. The library connects using unix socket with a
server part and asks this server about each file operation. The server can
say to permit or to block an operation.
2) The server part, which logs requests from clients and answers them.

The integration with the portage is made, using two hooks on
build(EbuildBuild.py):
 build = EbuildExecuter(background=self.background, pkg=pkg,
  scheduler=scheduler, settings=settings)

 build.addStartListener(self._build_start)
 build.addExitListener(self._build_stop)

 self._start_task(build, self._build_exit)

The _build_start creates the server thread, and passes a unix-socket path
using LOG_SOCKET environment variable.

 self.logserver.start()

 # Copy socket path to LOG_SOCKET environment variable
 env=self.settings.configdict["pkg"]
 env['LOG_SOCKET'] = self.logserver.socket_name

The  _build_stop stops the server and removes LOG_SOCKET environment
variable.
 env=self.settings.configdict["pkg"]
 if 'LOG_SOCKET' in env:
  del env['LOG_SOCKET']

 events=self.logserver.stop()
 self.logserver=None

If LOG_SOCKET env var is set then the new processes(like ebuild.sh <stage>)
are started with my library injected using LD_PRELOAD trick.

The problem: on newer portage versions packages are merged in a subprocess
to allow the Scheduler run in main thread while files are moved or copied
asynchronously(file _MergeProcess.py). This subprocess is created before
the _build_stop is called, so it still has a copy of settings with
LOG_SOCKET set. When _build_stop is called the server stops and merging
fails because the unix-socket is closed.

To solve this I added this code before forking in _MergeProcess.py:
 # Delete LOG_SOCKET from environment
 env=self.settings.configdict["pkg"]
 if 'LOG_SOCKET' in env:
  del env['LOG_SOCKET']

But I think this can be done better. Do you know a better way?

P.S. The old variant of integration patch is available here:
http://git.overlays.gentoo.org/gitweb/?p=proj/autodep.git;a=blob;f=integration_with_portage.patch;h=ff8f487306e85cbc6ba9e69be6fe613e99b39b53;hb=HEAD

Best,
Alexander Bersenev

2014-02-14 14:08 GMT+06:00 Александр Берсенев <bay@×××××××××.ru>:

> Aleksandr
>
>
> 2014-02-14 13:50 GMT+06:00 Greg Turner <gmt@×××××.us>:
>
> Holy crap, that looks awesome!  How does one pronounce your name,
>> Александр?
>>
>> On Thu, Feb 13, 2014 at 11:28 PM, Александр Берсенев <bay@×××××××××.ru>
>> wrote:
>> > Ok, I'll work on it.
>> >
>> >
>> > 2014-02-13 23:00 GMT+06:00 Rich Freeman <rich0@g.o>:
>> >
>> >> On Thu, Feb 13, 2014 at 11:07 AM, Brian Dolbec <dolsen@g.o>
>> wrote:
>> >> > Yes, if you can please work on updating it.  Please contact us on the
>> >> > gentoo-portage-dev mail list or irc #gentoo-portage for changes to
>> >> > portage that will help keep it working in the future.  I started
>> >> > development of a public_api branch long ago just for having a stable
>> >> > API for apps to use.  Perhaps some of what you need can be integrated
>> >> > there.
>> >>
>> >> Seems like a valuable tool.  It would be best if it could either use
>> >> portage APIs where available, or if effort could be directed at
>> >> incorporating the necessary features into portage or its APIs so that
>> >> you're not stuck maintaining a fork.  I'm sure the portage team will
>> >> help where they can.
>> >>
>> >> Rich
>> >>
>> >
>>
>>
>