Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.

[Richard Yao]

On 07/01/2013 09:56 PM, Greg KH wrote:> On Mon, Jul 01, 2013 at
09:36:21PM -0400, Richard Yao wrote:
>> On 07/01/2013 03:23 PM, Greg KH wrote:
>>> On Mon, Jul 01, 2013 at 08:45:16PM +0200, Tom Wijsman wrote:
>>>>>> Q: What about my stable server? I really don't want to run this
>>>>>> stuff!
>>>>>>
>>>>>> A: These options would depend on !CONFIG_VANILLA or
>>>>>> CONFIG_EXPERIMENTAL
>>>>>
>>>>> What is CONFIG_VANILLA?  I don't see that in the upstream kernel tree
>>>>> at all.
>>>>>
>>>>> CONFIG_EXPERIMENTAL is now gone from upstream, so you are going to
>>>>> have a problem with this.
>>>>
>>>> Earlier I mentioned "2) These feature should depend on a non-vanilla /
>>>> experimental option." which is an option we would introduce under the
>>>> Gentoo distribution menu section.
>>>
>>> Distro-specific config options, great :(
>>>
>>>>>>    which would be disabled by default, therefore if you keep this
>>>>>> option the way it is on your stable server; it won't affect you.
>>>>>
>>>>> Not always true.  Look at aufs as an example.  It patches the core
>>>>> kernel code in ways that are _not_ accepted upstream yet.  Now you all
>>>>> are running that modified code, even if you don't want aufs.
>>>>
>>>> Earlier I mentioned "3) The patch should not affect the build by
>>>> default."; if it does, we have to adjust it to not do that, this is
>>>> something that can be easily scripted. It's just a matter of embedding
>>>> each + block in the diff with a config check and updating the counts.
>>>
>>> Look at aufs as a specific example of why you can't do that, otherwise,
>>> don't you think that the aufs developer(s) wouldn't have done so?
>>
>> I am accquainted with the developer of a stackable filesystem developer.
>> According to what he has told me in person offline, the developers on
>> the LKML cannot decide on how a stackable filesystem should be
>> implemented. I was told three different variations on the design that
>> some people liked and others didn't, which ultimately kept the upstream
>> kernel from adopting anything. I specifically recall two variations,
>> which were doing it as part of the VFS and doing it as part of ext4. If
>> you want to criticize stackable filesystems, would you lay out a
>> groundwork for getting one implemented upon which people will agree?
>
> I was not criticising stackable filesystems at all, nor the aufs code,
> which I personally run on some of my own systems.
>
> I agree that the upstream kernel development of stackable filesystems
> has been all over the place, with seemingly not much guidance on how to
> get things merged properly.  Being that I am not part of the subset of
> the kernel development community, I am in no position to lay any
> groundwork out at all.
>
> And note, this is totally off-topic from this thread.
>
> My main point is that if Gentoo wants to start maintaining out-of-tree
> kernel patches, and modifying them, the maintenance nightmare is going
> to be huge.  Been there, done that, have way too many t-shirts
> commemorating it, and never want to do it again.
>
>>> The goal of "don't touch any other kernel code" is a very good one, but
>>> not always true for these huge out-of-tree kernel patches.  Usually that
>>> is the main reason why these patches aren't merged upstream, because
>>> those changes are not acceptable.
>>
>> I was under the impression that there were several reasons for patches
>> not being merged upstream:
>>
>> 1. Lack of signed-off
>
> No distro will accept code that does not have a signed-off-by line,
> otherwise they might get into trouble, as that implies the patch is not
> properly licensed, right?

That is wrong. Signed-off is an affirmation that the code is under the
license, but the absence of signed-off does not imply that the code is
not under the license. For example, I doubt you will receive signed-off
for PaX/GrSecurity, but is there any doubt that code is under the GPL?

To give another example, I doubt that you will receive signed off for
code from BSD UNIX, but is there any doubt that code is under the
3-clause BSD license?

>> 2. Code drop that no one will maintain
>
> Agreed.
>
>> 3. Subsystem maintainers saying no simply because they do not like
>> <insert non-technical reason here>.
>
> That is very rare and usually the subsystem maintainer can be poked to
> resolve this.

Past conversations with you have made me reluctant to try, but the next
time I see something like this, I will send you an email.

>> 4. Risk of patent trolls
>
> I know of no kernel patches outside of the tree because of this, do you?

There is compatibility code for DOS long filenames in fat that is no
longer in-tree because of this.

>> 5. Actual technical reasons
>
> That's the majority of the reasons patches aren't accepted.

Not all patches that aren't accepted are submitted to be rejected.

>> Only some of the patches were rejected. Others were never submitted. The
>> PaX/GrSecurity developers prefer their code to stay out-of-tree. As one
>> of the people hacking on ZFSOnLinux, I prefer that the code be
>> out-of-tree.
>
> There's also a small legal issue with the zfs code that might prevent it
> from being merged :)

To summarize an email from Linus, the reason people who want ZFS in the
main tree cannot have that is his signed-off policy. At least one of the
copyright holders has no interest in providing it.

>> That is because fixes for other filesystems are either held back by a
>> lack of system kernel updates or held hostage by regressions in newer
>> kernels on certain hardware.
>
> I have never heard this being a reason for keeping code upstream, what
> do you mean by it?

This is an issue that end users tend to encounter where changes in a
newer kernel break stuff that they need. One example is nouveau kexec
support. Another is that the nouveau in the first two RCs of Linux 3.7
(if I recall) broke my display completely.

>> With that said, being in Linus' tree does not make code fall under some
>> golden standard for quality. There are many significant issues in code
>> committed to Linus' the kernel, some of which have been problems for
>> years. Just to name a few:
>
> <bug reports snipped>
>
> The fact that bugs happen in 16 million lines of kernel code, moving at
> a rate that is orders of magnitude faster than any other software
> development project, is not news to anyone, it's a fact of life.
>
> The fact that we fix them when they are found is the important thing,
> don't you agree?
>
> And of course, any recommendations on how we can find these bugs before
> they are merged is _always_ accepted.

My point is that code in Linus' tree is not free from error. It has bugs
too and is by no means always better than out-of-tree code.

>> Being outside Linus' tree is not synonymous with being bad and being bad
>> is not synonymous with being rejected. It is perfectly reasonable to
>> think that there are examples of good code outside Linus' tree.
>
> Being outside of Linus's tree puts the maintenance and support burden on
> you, and you are working against a body of code that is going faster
> this week than it did last week, so your work is constantly increasing.
> That is why it should be merged, to save yourself work.

Then you get to be a kernel subsystem maintainer and you get to check
which version of the code was included in the kernel for each bug
report, rather than just having one codebase.

>> Furthermore, should the kernel kernel choose to engage that out-of-tree
>> code, my expectation is that its quality will improve as they do testing
>> and write patches.
>
> What do you mean by this?
>
> thanks,
>
> greg k-h
>

I started using ZFSOnLinux last year. Since then, I have written several
dozen patches. I wrote support for multiple new kernels, wrote support
for PaX/GrSecurity, diagnosed why swap on zvols was broken and wrote the
initial patches to fix it, collaborated on kernel preemption support,
fixed several kernel panics, etcetera. Nearly all of that work has been
merged by ZFSOnLinux upstream, has been rejected under mutual agreement
or is pending under review. If Gentoo developers begin maintaining other
out-of-linus'-tree kernel code in Gentoo, I imagine that similar things
would happen with that code as well.