1 Bionic comes with a set of 'clean' Linux kernel headers that can safely be
2 included by userland applications and libraries without fear of hideous
3 conflicts. for more information why this is needed, see the "RATIONALE"
4 section at the end of this document.
6 these clean headers are automatically generated by several scripts located
7 in the 'bionic/kernel/tools' directory, which process a set of original
8 and unmodified kernel headers in order to get rid of many annoying
9 declarations and constructs that usually result in compilation failure.
11 the 'clean headers' only contain type and macro definitions, with the
12 exception of a couple static inline functions used for performance
13 reason (e.g. optimized CPU-specific byte-swapping routines)
15 they can be included from C++, or when compiling code in strict ANSI mode.
16 they can be also included before or after any Bionic C library header.
18 the generation process works as follows:
20 * 'bionic/kernel/original/'
21 contains a set of kernel headers as normally found in the 'include'
22 directory of a normal Linux kernel source tree. note that this should
23 only contain the files that are really needed by Android (use
24 'find_headers.py' to find these automatically).
26 * 'bionic/kernel/common'
27 contains the non-arch-specific clean headers and directories
28 (e.g. linux, asm-generic and mtd)
30 *'bionic/kernel/arch-arm/'
31 contains the ARM-specific directory tree of clean headers.
33 * 'bionic/kernel/arch-arm/linux'
34 is really a symlink to '../common/linux'. ditto for the 'asm-generic'
35 and 'mtd' directories
37 * 'bionic/kernel/arch-arm/asm'
38 contains the real ARM-specific headers
40 * 'bionic/kernel/arch-x86'
41 similarly contains all headers and symlinks to be used on x86
43 * 'bionic/kernel/tools' contains various Python and shell scripts used
44 to manage and re-generate the headers
46 the tools you can use are:
48 * tools/find_users.py
49 scans a list of source files or directories and prints which ones do
50 include Linux headers.
52 * tools/find_headers.py
53 scans a list of source files or directories and recursively finds all
54 the original kernel headers they need.
56 * tools/clean_header.py
57 prints the clean version of a given kernel header. with the -u option,
58 this will also update the corresponding clean header file if its
59 content has changed. you can also process more than one file with -u
61 * tools/update_all.py
62 automatically update <linux/_config.h> and all clean headers from
63 the content of 'bionic/kernel/original'. this is the script you're
64 likely going to run whenever you update the original headers.
66 NOTE:
67 if ANDROID_PRODUCT_OUT is defined in your environment, both 'clean_header.py'
68 and 'update_all.py' will automatically issue "p4 add/edit/delete" commands
69 appropriately to reflect the changes being made.
71 you will need to "p4 submit" manually though...
74 HOW TO BUILD BIONIC AND OTHER PROGRAMS WITH THE CLEAN HEADERS:
75 ==============================================================
77 simply add bionic/kernel/arch-<yourarch> to your C include path. that should
78 be enough. Note that Bionic will not compile properly if you don't.
81 HOW TO SUPPORT ANOTHER ARCHITECTURE:
82 ====================================
84 see the content of tools/defaults.py, you will need to make a few updates
85 here:
87 - add a new item to the 'kernel_archs' list of supported architectures
89 - add a proper definition for 'kernel_known_<arch>_statics' with
90 relevant definitions.
92 - update 'kernel_known_statics' to map "<arch>" to
93 'kernel_known_<arch>_statics'
95 then, add the new architecture-specific headers to original/asm-<arch>.
96 (please ensure that these are really needed, e.g. with tools/find_headers.py)
98 finally, run tools/update_all.py
102 HOW TO UPDATE THE HEADERS WHEN NEEDED:
103 ======================================
105 IMPORTANT IMPORTANT:
107 WHEN UPDATING THE HEADERS, ALWAYS CHECK THAT THE NEW CLEAN HEADERS DO
108 NOT BREAK THE KERNEL <-> USER ABI, FOR EXAMPLE BY CHANGING THE SIZE
109 OF A GIVEN TYPE. THIS TASK CANNOT BE EASILY AUTOMATED AT THE MOMENT
111 copy any updated kernel header into the corresponding location under
112 'bionic/kernel/original'.
114 for any new kernel header you want to add, first run tools/find_headers.py to be
115 sure that it is really needed by the Android sources. then add it to
116 'bionic/kernel/original'
118 then, run tools/update_all.py to re-run the auto-cleaning
122 HOW THE CLEANUP PROCESS WORKS:
123 ==============================
125 this section describes the action performed by the cleanup program(s) when they
126 process the original kernel headers into clean ones:
128 1. Optimize well-known macros (e.g. __KERNEL__, __KERNEL_STRICT_NAMES)
130 this pass gets rid of everything that is guarded by a well-known macro
131 definition. this means that a block like
133 #ifdef __KERNEL__
134 ....
135 #endif
137 will be totally omitted from the output. the optimizer is smart enough to
138 handle all complex C-preprocessor conditional expression appropriately.
139 this means that, for example:
141 #if defined(__KERNEL__) || defined(FOO)
142 ...
143 #endif
145 will be transformed into:
147 #ifdef FOO
148 ...
149 #endif
151 see tools/defaults.py for the list of well-known macros used in this pass,
152 in case you need to update it in the future.
154 note that this also remove any reference to a kernel-specific configuration
155 macro like CONFIG_FOO from the clean headers.
158 2. remove variable and function declarations:
160 this pass scans non-directive text and only keeps things that look like a
161 typedef or struct declaration. this allows to get rid of any variable or
162 function declaration that should only be used within the kernel anyway
163 (and which normally *should* be guarded in a #ifdef __KERNEL__ ... #endif
164 block, if the kernel writers were not so messy)
166 ther are however a few exceptions: it is seldom useful to keep the definition
167 of some static inline functions performing very simple operations. a good
168 example is the optimized 32-bit byte-swap function found in
169 arch-arm/asm/byteorder.h
171 the list of exceptions is in tools/defaults.py in case you need to update it
172 in the future.
174 note that we do *not* remove macro definitions, including these macro that
175 perform a call to one of these kernel-header functions, or even define other
176 functions. we consider it safe since userland applications have no business
177 using them anyway.
180 4. whitespace cleanup:
182 the final pass remove any comments and empty lines from the final headers.
185 5. add a standard disclaimer:
187 prepended to each generated header, contains a message like
188 "do not edit directly - file was auto-generated by ...."
191 RATIONALE:
192 ==========
194 OVERVIEW OF THE CURRENT KERNEL HEADER MESS:
195 -------------------------------------------
197 The original kernel headers are not easily usable from userland applications.
198 they contain many declarations and construct that will result in a compilation
199 failure or even worse, incorrect behaviour. for example:
201 - some headers try to define Posix types (e.g. size_t, ssize_t) that can
202 conflict with the corresponding definitions provided by your C library.
204 - some headers use constructs that cannot be compiled in ANSI C mode.
206 - some headers use constructs do not compile with C++ at all.
208 - some headers contain invalid "legacy" definitions for the benefit of old
209 C libraries (e.g. glibc5) but result in incorrect behaviour if used
210 directly.
212 e.g. gid_t being defined in <linux/types.h> as a 16-bit type while
213 __kernel_gid_t is 32-bit. this results in problems when getgroups() or
214 setgroups() are called, since they operate on gid_t arrays.
216 unfortunately, these headers are also the only source of some really extensive
217 constant and type definitions that are required by userland applications.
218 think any library/program that need to access ALSA, or Video4Linux, or
219 anything related to a specific device or Linux-specific system interface
220 (e.g. IOCTLS, etc...)
222 As a consequence, every Linux distribution provides a set of patched kernel
223 headers to be used by userland applications (which installs in
224 /usr/include/linux/, /usr/include/asm/, etc...). these are manually maintained
225 by distribution packagers, and generated either manually or with various
226 scripts. these headers are also tailored to GNU LibC and cannot be reused
227 easily by Bionic.
229 the kernel authors have already stated that they don't want to fix the
230 problem, even when someone proposed a patch to start cleaning the official
231 headers. from their point of view this is purely a library author problem.
233 yeah, right....
236 WHAT WE DO:
237 -----------
239 so we're doomed to repeat the same effort than anyone else. the big difference
240 here is that we want to automate as much as possible the generation of the
241 clean headers to easily support additional architectures in the future,
242 and keep current with upstream changes in the header definitions with the
243 least possible hassle.
245 of course, this is only a race to the bottom. the kernel maintainers still
246 feel free to randomly break the structure of their headers (e.g. moving the
247 location of some files) occasionally, so we'll need to keep up with that by
248 updating our build script/original headers as these cases happen.
250 what we do is keep a set of "original" kernel headers, and process them
251 automatically to generate a set of "clean" headers that can be used from
252 userland and the C library.
254 note that the "original" headers can be tweaked a little to avoid some subtle
255 issues. for example:
257 - when the location of various USB-related headers changes in the kernel
258 source tree, we want to keep them at the same location in our generated
259 headers (there is no reason to break the userland API for something
260 like that).
262 - sometimes, we prefer to take certain things out of blocks guarded by a
263 #ifdef __KERNEL__ .. #endif. for example, on recent kernels <linux/wireless.h>
264 only includes <linux/if.h> when in kernel mode. we make it available to
265 userland as well since some code out there assumes that this is the case.
267 - sometimes, the header is simply incorrect (e.g. it uses a type without
268 including the header that defines it before-hand)