Re: [gentoo-soc] zfs-kmod was Application for Google Summer of Code 2018-Mishal Roy

Re: [gentoo-soc] zfs-kmod was Application for Google Summer of Code 2018-Mishal Roy

[Rich Freeman]

On Fri, Mar 23, 2018 at 9:49 AM, EBo <ebo@sandien.com> wrote:
> On Mar 23 2018 7:37 AM, Rich Freeman wrote:
>>
>> On Fri, Mar 23, 2018 at 2:50 AM, Patrick Lauer <patrick@gentoo.org> wrote:
>>>
>>>
>>> https://github.com/adjust/gentoo-overlay/tree/master/sys-kernel
>>>
>>> This just needs to be cleaned up a bit to be upstreamed, and so far I've
>>> not had the time for it.
>>>
>>> It's a solved problem (I'm aware of at least two other ebuilds to do the
>>> same) :)
>>>
>>
>> Yeah, that is certainly one approach.  Part of me thinks that
>> modularizing the initramfs portion would be ideal, and as Benda
>> suggested making it possible to install the built sources might be
>> useful (if doing so one would need to patch the file that points to
>> the build location so that future module builds can find them).  This
>> particular ebuild uses sources in /usr/src but I think it would be
>> cleaner to just fetch these into the build directory.
>>
>> The package I find most frustrating is zfs-kmod, because it needs
>> fully-built sources and not just prepared sources.  I build to a
>> tmpfs, and re-creating prepared sources using that approach is
>> trivial, but doing a full rebuild just to update zfs modules is a bit
>> painful (granted, less so now that I have 12 cores).  I've been too
>> lazy to figure out why zfs needs them...
>
> In the past I have had zfs-kmod seriously break my kernel builds.  Cleaning
> that up to be consistent would probably be useful to the general community
> as a whole IMNSHO.  Isn't there a way to set the build and install
> directories from some env variable?

Can you elaborate as to what exactly happened?  I wouldn't think that
zfs-kmod could touch your kernel sources due to the sandbox since it
is built using portage.  A quick check shows that all the files it
installs seem unlikely to collide with anything you should already
have.

When you're building a kernel you can certainly direct its output, but
that is before zfs-kmod even enters into things.  I typically do this
with something like:
cd /usr/src/linux
make O=/var/tmp/linux oldconfig    (and so on)

Then if you emerge zfs-kmod it would use the kernel sources in
/usr/src/linux and build output in /var/tmp/linux to do its own build,
assuming both still exist.

-- 
Rich

Re: [gentoo-soc] zfs-kmod was Application for Google Summer of Code 2018-Mishal Roy

[EBo]

On Mar 23 2018 8:50 AM, Rich Freeman wrote:
> On Fri, Mar 23, 2018 at 9:49 AM, EBo <ebo@sandien.com> wrote:
>> On Mar 23 2018 7:37 AM, Rich Freeman wrote:
>>>
>>> On Fri, Mar 23, 2018 at 2:50 AM, Patrick Lauer <patrick@gentoo.org> 
>>> wrote:
>>>>
>>>>
>>>> https://github.com/adjust/gentoo-overlay/tree/master/sys-kernel
>>>>
>>>> This just needs to be cleaned up a bit to be upstreamed, and so 
>>>> far I've
>>>> not had the time for it.
>>>>
>>>> It's a solved problem (I'm aware of at least two other ebuilds to 
>>>> do the
>>>> same) :)
>>>>
>>>
>>> Yeah, that is certainly one approach.  Part of me thinks that
>>> modularizing the initramfs portion would be ideal, and as Benda
>>> suggested making it possible to install the built sources might be
>>> useful (if doing so one would need to patch the file that points to
>>> the build location so that future module builds can find them).  
>>> This
>>> particular ebuild uses sources in /usr/src but I think it would be
>>> cleaner to just fetch these into the build directory.
>>>
>>> The package I find most frustrating is zfs-kmod, because it needs
>>> fully-built sources and not just prepared sources.  I build to a
>>> tmpfs, and re-creating prepared sources using that approach is
>>> trivial, but doing a full rebuild just to update zfs modules is a 
>>> bit
>>> painful (granted, less so now that I have 12 cores).  I've been too
>>> lazy to figure out why zfs needs them...
>>
>> In the past I have had zfs-kmod seriously break my kernel builds.  
>> Cleaning
>> that up to be consistent would probably be useful to the general 
>> community
>> as a whole IMNSHO.  Isn't there a way to set the build and install
>> directories from some env variable?
>
> Can you elaborate as to what exactly happened?  I wouldn't think that
> zfs-kmod could touch your kernel sources due to the sandbox since it
> is built using portage.  A quick check shows that all the files it
> installs seem unlikely to collide with anything you should already
> have.
>
> When you're building a kernel you can certainly direct its output, 
> but
> that is before zfs-kmod even enters into things.  I typically do this
> with something like:
> cd /usr/src/linux
> make O=/var/tmp/linux oldconfig    (and so on)
>
> Then if you emerge zfs-kmod it would use the kernel sources in
> /usr/src/linux and build output in /var/tmp/linux to do its own 
> build,
> assuming both still exist.

It has been quite some time since this has happened (I seem to remember 
removing zfs from that machine due to the headaches, but I would have to 
check as it is possible that someone just fixed it and it now works), 
but as I recall every time that I built a new kernel with genkernel that 
it then runs 'emerge @module-rebuild' automatically before the install.  
If that fails due to some conflict or dependency, then everything gets 
jammed up.  Yes, if you are building the kernel by hand, then you *may* 
be able to ignore a broken module, but as I recall in those instances it 
would not finish the build and install.

At this point I would have to try to replicate the scenario.  I still 
have that machine around and could do this, but the timing is a bit bad 
for me.  Do let me know if you need this replicated and give me some 
time lines so that I can figure out if I can do this by the time you 
need.

   Thanks and best regards,

   EBo --