Gentoo Archives: gentoo-portage-dev

From: Joakim Tjernlund <joakim.tjernlund@×××××××××.se>
To: "gentoo-portage-dev@l.g.o" <gentoo-portage-dev@l.g.o>
Subject: Re: [gentoo-portage-dev] Re: running ebuild in src tree
Date: Thu, 12 Mar 2015 17:27:03
Message-Id: 1426181219.31989.128.camel@transmode.se
In Reply to: [gentoo-portage-dev] Re: running ebuild in src tree by Duncan <1i5t5.duncan@cox.net>
1 On Thu, 2015-03-12 at 01:27 +0000, Duncan wrote:
2 > Zac Medico posted on Wed, 11 Mar 2015 12:03:10 -0700 as excerpted:
3 >
4 > > On 03/11/2015 11:56 AM, Joakim Tjernlund wrote:
5 > > > On Wed, 2015-03-11 at 11:34 -0700, Zac Medico wrote:
6 > > > > On 03/11/2015 09:03 AM, Joakim Tjernlund wrote:
7 > > > > > When developing code it would be really nice if one could run your ebuild on that src tree as is(no
8 > > > > > fetch, unpack etc.)
9 > > > >
10 > > > > The existing convention is to create an ebuild with version 9999 and use one of the live vcs eclasses
11 > > > > such as git-r3 to pull the live sources in the src_unpack function. In a future EAPI, we plan to add
12 > > > > some features related to this [1].
13 > > >
14 > > > I think you misunderstand, [1] is not what I want to do(I think):
15 > > >
16 > > > Got my src working copy and made a few modds, not commitet yet. Now I just want build/test etc. before
17 > > > committing and to do that I just run mytree/overlay/dev-util/myapp/myapp.ebuild compile and voila, my
18 > > > code is built which I already have in mytree.
19 > >
20 > > Well, you can create a -9999 ebuild that copies your sources from $directory to $WORKDIR. Maybe use an
21 > > environment to configure whether it pulls from a local directory or a vcs repository.
22 >
23 > @ Joakim T:
24 >
25 > FWIW, a commonly recommended user-level portage optimization is to point
26 > $PORTAGE_TMPDIR at a tmpfs mount. As long as you have sufficient memory,
27 > that lets all building take place in the tmpfs and thus in memory, eliminating many read-accesses and
28 > most/all write accesses to permanent storage during the build and (fake-)install phases.
29 >
30 > In addition to speeding up emerge itself, this reduces build-time I/O, which often becomes the bottleneck
31 > on which other processes may be waiting as well, thus allowing other processes more efficient access to
32 > permanent storage while emerge is ongoing. Between this and setting PORTAGE_NICENESS=20, emerge is /much/
33 > better behaved during builds, interrupting other processes much less and thus letting you carry on with
34 > other things while emerge is doing its thing, with far less interruption.
35 > =:^)
36 >
37 > For instance, here I have /tmp as a tmpfs mount (with /var/tmp being a bind-mount of the same tmpfs), and
38 > in make.conf, have the line:
39 >
40 > PORTAGE_TMPDIR=/tmp
41 >
42 > Emerge then creates /tmp/portage, and within it, creates the usual cat/ pkg/ build trees, etc, as it
43 > emerges various packages.
44 >
45 >
46 > Obviously, your sources in permanent storage are going to be cache-copied
47 > into memory as you do the build anyway, and pointing PORTAGE_TMPDIR at tmpfs then becomes a copy to
48 > (tmpfs) memory only. While that doesn't technically eliminate the copies (since the read into tmpfs will
49 > cache and you'll have the tmpfs copy as well), it DOES mean most of the work beyond the initial read into
50 > memory will be memory-only, so you DO eliminate the permanent-storage copies.
51 >
52 > Is that sufficiently close to what you were looking to do? Beyond that,
53 > as Zac suggests, just have the ebuild grab the sources from wherever you
54 > put them as your src_unpack, as at that point it'll be a copy to tmpfs, and thus take essentially the same
55 > time (or even less since it'll avoid the build-time writes to permanent storage) as doing the in-place
56 > build directly. Plus, creating a tmpfs mount if necessary, and setting PORTAGE_TMPDIR, is easy, and
57 > you'll dramatically speed-up normal builds as well. =:^)
58 >
59
60 No, there can be no copy of sources for what I want. It just gets in the way having to do that.
61 Hacks like this seems to work:
62
63 post_src_compile() {
64 # make it compile every time
65 rm -f ${PORTAGE_BUILDDIR}/.compiled
66 }
67
68 post_src_install() {
69 # make it install every time
70 rm -f ${PORTAGE_BUILDDIR}/.installed
71 }
72
73 #hmm, doesn't seem to be an post_package function
74 #post_package() {
75 # rm -f ${PORTAGE_BUILDDIR}/.packaged
76 #}
77
78 src_unpack() {
79 #dir need to exist
80 mkdir -p ${S} || die
81 }
82 src_compile() {
83 EBUILDDIR=$(dirname ${FILESDIR})
84 MYTRUNK=${EBUILDDIR}/../../..
85 # add sandbox exceptions
86 addwrite ${MYTRUNK}
87
88 cd ${MYTRUNK} || die
89 cd "${PN}"
90 .....
91 }

Replies

Subject Author
[gentoo-portage-dev] Re: running ebuild in src tree Duncan <1i5t5.duncan@×××.net>
Re: [gentoo-portage-dev] Re: running ebuild in src tree Alec Warner <antarus@g.o>