1 |
Hi, |
2 |
|
3 |
I've submitted a proposal for kernelconfig today, |
4 |
an attempt at creating kernel configurations automatically, |
5 |
with hardware detection and package management integration, |
6 |
included in this mail. Feel free to comment ;-) |
7 |
|
8 |
Kind Regards, |
9 |
André Erdmann |
10 |
|
11 |
|
12 |
=== proposal starts here === |
13 |
|
14 |
|
15 |
|
16 |
Abstract |
17 |
======== |
18 |
|
19 |
Automated creation of kernel configuration files |
20 |
from curated sources, detected hardware, installed packages and user input. |
21 |
|
22 |
|
23 |
|
24 |
Objective |
25 |
========= |
26 |
|
27 |
Kernel configuration can be a tiresome procedure. |
28 |
One starts with a basic config file, |
29 |
either generated, downloaded or taken from previous runs, |
30 |
and has to know - or find out - which options need to be set |
31 |
for a particular piece of hard- or software. |
32 |
|
33 |
This project aims at automating kernel configuration in both areas. |
34 |
For hardware, this involves detecting devices, possibly on a different machine, |
35 |
and mapping them to kconfig options. |
36 |
And for software, this means setting a number of options in order to fit |
37 |
a specific role determined by both installed packages and user preferences. |
38 |
|
39 |
Targeting at beginners and more experienced users, |
40 |
it is necessary to leave choices up to users while filling the gaps |
41 |
via automatisms. |
42 |
For example, "package qemu with kvm and without networking" is more specific |
43 |
than "virtualization", but both may or may not result in the same configuration, |
44 |
depending on architecture/hardware, installed packages and other user choices. |
45 |
|
46 |
To realize this, a more flexible configuration mini-language will be introduced, |
47 |
based on and compatible with the existing facilities. |
48 |
|
49 |
At last, it should be possible to detect whether the resulting configuration |
50 |
makes sense and whether it is conflict-free. |
51 |
|
52 |
|
53 |
|
54 |
Ideas |
55 |
===== |
56 |
|
57 |
This section gives a more in-depth view of what I have planned. |
58 |
|
59 |
|
60 |
Basic Design |
61 |
------------ |
62 |
|
63 |
Users define a configuration basis and an output state, and kernelconfig |
64 |
computes how to get there. |
65 |
|
66 |
The configuration basis may stem from a command ("make someconfig"), |
67 |
a curated source, or any other file. |
68 |
|
69 |
The output state is what users declare in the settings file or via command |
70 |
line parameters. It consists of direct instructions where config options |
71 |
are given by name, e.g. enable CONFIG_E1000E, |
72 |
and indirect instructions where a symbolic name |
73 |
such as "package shorewall" needs to be resolved to actual config |
74 |
options and their acceptable values. |
75 |
|
76 |
In either case, the instructions are incomplete, |
77 |
because the explicitly stated config options depend on other options, |
78 |
which in turn depend on even more options. |
79 |
Requiring users to write full configuration fragments in the settings file |
80 |
is highly impractical, so one of kernelconfig's essential tasks is |
81 |
detecting kconfig symbol dependencies |
82 |
and resolving the dependency graph (if possible). |
83 |
|
84 |
Another issue is that human input is prone to errors |
85 |
like assigning a string value to a tristate-type kconfig option |
86 |
or making two different decisions for a kconfig choice, |
87 |
for example setting both 'cfq' and 'deadline' as the default I/O scheduler. |
88 |
kernelconfig will detect and handle such errors. |
89 |
|
90 |
|
91 |
Configuration Basis |
92 |
------------------- |
93 |
|
94 |
The base configuration file does not necessarily exist on the machine |
95 |
running kernelconfig, it has to be obtained somehow. |
96 |
The "somehow" may be expressed in two ways: |
97 |
|
98 |
* a script/executable responsible for getting the configuration basis |
99 |
|
100 |
* a source definition ini-like file that describes how to get the |
101 |
configuration basis, but leaves the actual work up to kernelconfig |
102 |
|
103 |
The preferred variant for curated sources will be the source definition |
104 |
file, because it allows kernelconfig to verify user input, for example |
105 |
whether the architecture is supported by a particular curated source |
106 |
or whether a parameter is known and valid. |
107 |
A combination of both source definition file and script can be used |
108 |
for more complex sources. |
109 |
|
110 |
|
111 |
Macros |
112 |
------ |
113 |
|
114 |
Macros are instructions that modify the kernel configuration. |
115 |
Users list them in the "[options]" section of the settings file |
116 |
and may include other macro files, so-called feature sets. |
117 |
|
118 |
It is planned to implement a mini-language with the following features: |
119 |
|
120 |
* additional actions, |
121 |
for example "enable-or-module", "append-string", "include feature-set" |
122 |
|
123 |
* disable macros if certain conditions are not met ("<macro> if <condition(s)>") |
124 |
|
125 |
* parameters, mostly for controlling how feature sets are imported |
126 |
|
127 |
|
128 |
Hardware Detection |
129 |
------------------ |
130 |
|
131 |
Hardware detection can be viewed as a feature set that consists |
132 |
of conditional macros: enable one or more config options if a certain |
133 |
hardware identifier is matched:: |
134 |
|
135 |
enable-or-module I915 if modalias-match pci:<id_A> or pci:<id_B> |
136 |
|
137 |
The difficulty here is |
138 |
creating the hardware identifier to kernel config options mapping. |
139 |
It is currently planned to examine the "modules.alias" file, |
140 |
which maps sysfs modalias entries to kernel module names. |
141 |
The module names have to be translated back into config options by partially |
142 |
parsing the module Makefiles of the kernel sources. |
143 |
|
144 |
This is a (run)time-expensive task, |
145 |
"modules.alias" is generated by depmod and requires to build all kernel |
146 |
modules, ideally configured with "make allmodconfig", |
147 |
so kernelconfig will offer a precomputed file. |
148 |
|
149 |
|
150 |
Package Management Integration |
151 |
------------------------------ |
152 |
|
153 |
Two approaches are being considered: |
154 |
|
155 |
* a backwards-compatible one that greps through logs and vdb/ebuilds |
156 |
|
157 |
This creates a dynamic feature set |
158 |
(its content has to be calculated at runtime). |
159 |
|
160 |
* feature set catalog, |
161 |
a directory structure containing feature set files that can be included |
162 |
in the settings file (single packages or as a whole) |
163 |
|
164 |
Packages may either generate a feature set file from linux-info.eclass's |
165 |
CONFIG_CHECK variable, or install their own feature set files, |
166 |
which is more reliable and cross-distro friendly. |
167 |
|
168 |
This approach involves writing a new eclass, |
169 |
responsible for installing feature set files |
170 |
and converting CONFIG_CHECK. |
171 |
|
172 |
|
173 |
Genkernel Integration |
174 |
--------------------- |
175 |
|
176 |
to be discussed, |
177 |
but probably a facility in genkernel that allows to use kernelconfig |
178 |
instead of genkernel's gen_configkernel.sh->config_kernel(). |
179 |
|
180 |
|
181 |
|
182 |
Deliverables |
183 |
============ |
184 |
|
185 |
kernelconfig will be mostly written in Python, with smaller parts |
186 |
like hardware identifier collection (scan through sysfs) in Shell. |
187 |
Documentation will be written in reStructuredText. |
188 |
|
189 |
|
190 |
Final (August 15) |
191 |
----------------- |
192 |
|
193 |
A kernelconfig python script + modules with the following properties: |
194 |
|
195 |
* has access to various curated sources |
196 |
|
197 |
* processes macros from the settings file as described below (see |
198 |
`Mid-term`_) |
199 |
|
200 |
* has access to package information (`Package Management Integration`_) |
201 |
|
202 |
* maps detected hardware to config options |
203 |
|
204 |
* can also be called via genkernel |
205 |
|
206 |
|
207 |
A shell script for detecting hardware, |
208 |
with the possibility to run it on one machine (without requiring Python) |
209 |
and use the reported data on another machine. |
210 |
|
211 |
An ebuild for installing kernelconfig, |
212 |
though it may also be used standalone. |
213 |
|
214 |
|
215 |
Documentation: |
216 |
|
217 |
* developer documentation for all features |
218 |
|
219 |
* end-user documentation: man pages and usage guide |
220 |
|
221 |
|
222 |
Mid-term (June 20) |
223 |
------------------ |
224 |
|
225 |
A kernelconfig python script + modules with the following properties: |
226 |
|
227 |
* uses a .config file as configuration basis |
228 |
|
229 |
* processes macros from the settings file: |
230 |
|
231 |
* checks whether kconfig options are known |
232 |
|
233 |
* type-checks options/values |
234 |
|
235 |
* detects kconfig symbol dependencies + conflicts, and finds a solution |
236 |
|
237 |
* feature sets mechanism (loading of macro files) |
238 |
|
239 |
* simple unit test cases around kconfig symbol dependency resolution |
240 |
|
241 |
|
242 |
Documentation: partially, |
243 |
|
244 |
* developer documentation mostly complete for "basic design" and |
245 |
"macro syntax" |
246 |
|
247 |
* end-user documentation: usage guide: "basic program usage", |
248 |
"settings file / macro syntax" |
249 |
|
250 |
|
251 |
|
252 |
Timeline |
253 |
======== |
254 |
|
255 |
May 23 - May 29: write initial documentation: "basic design", "macro syntax" |
256 |
|
257 |
May 30 - Jun 08: implement "basic design" and "macro syntax", |
258 |
but without dependency resolution (yet) |
259 |
|
260 |
Jun 09 - Jun 19: add kconfig dependency resolution |
261 |
|
262 |
Jun 20 - Jun 26: Mid-term evaluations / write documentation |
263 |
|
264 |
Jun 27 - Jul 10: (re-)add configuration sources |
265 |
|
266 |
Jul 11 - Jul 17: add hardware detection |
267 |
|
268 |
Jul 18 - Jul 24: add package management integration |
269 |
|
270 |
Jul 25 - Jul 31: add genkernel integration |
271 |
|
272 |
Aug 01 - Aug 23: testing, write/improve documentation |
273 |
|
274 |
|
275 |
|
276 |
Biography / About me |
277 |
==================== |
278 |
|
279 |
I'm a twenty-four year old student from Stuttgart, Germany. |
280 |
I've been using Gentoo for quite a while now and participated in GSoC |
281 |
with Gentoo twice before, in 2012 and 2013, which was centered around |
282 |
roverlay, an effort on generating an overlay for R packages. |
283 |
This would be my third and final summer of code. |
284 |
|
285 |
My most long-term open source contribution is doing reviews for TLP, |
286 |
a shell-script based power management tool, since 2011. |
287 |
I occasionally contribute to other projects, too, for example PyUSB. |
288 |
|
289 |
|
290 |
|
291 |
Extra Information |
292 |
================= |
293 |
|
294 |
Use the tools that you will use in your project to make changes to code |
295 |
----------------------------------------------------------------------- |
296 |
|
297 |
My most recent activity on bugs.gentoo.org is bug #574782, |
298 |
https://bugs.gentoo.org/show_bug.cgi?id=574782. |
299 |
|
300 |
Only slightly related, I'm a bit familiar with genkernel-next, |
301 |
which is based on genkernel, see genkernel-next #17, |
302 |
https://github.com/Sabayon/genkernel-next/issues/17. |
303 |
|
304 |
|
305 |
Participate in our development community |
306 |
---------------------------------------- |
307 |
|
308 |
You can find a mailing list entry from me at <this mail>. |
309 |
|
310 |
|
311 |
Contact info |
312 |
------------ |
313 |
|
314 |
email: dywi@×××××××.de |
315 |
irc: dywi at irc.freenode.net |
316 |
|
317 |
phone number: <removed> |
318 |
home mailing address: <removed> |
319 |
|
320 |
|
321 |
Working hours |
322 |
------------- |
323 |
|
324 |
Mo - Sa, 7 am - 8 pm UTC, |
325 |
with a few restrictions until Mid July due to classes |
326 |
|
327 |
I target an average of 35 hours per week. |