[gentoo-soc] Gentoo on Android: Final Report

Gentoo on Android: Final Report
                   ===============================

Author: Benda Xu
Date: 2013-09-25 10:55:24 JST


Table of Contents
=================
1 Overview
2 Toolchain
    2.1 Remaining Issue
    2.2 Credits
3 Profiles
    3.1 Credits
4 GLEP
    4.1 Credits
5 Bootstrap Script
    5.1 Credits
6 Stage 3 Tarball
    6.1 Remaining Issue
7 SD card partition
    7.1 Remaining Issue
8 Cross triplet-prefix a native toolchain
    8.1 Credits
9 Prefix Portage Escalation
10 Future
11 Final Acknowledgment


1 Overview 
===========
  The project is finished. A Gentoo RAP (:=Rap Ain't Prefix, also
  known as Prefix with libc) solution is developed and brought on to
  Android devices.
  

2 Toolchain 
============
  This is the central part of the RAP magic. The final solution is to
  use the sysroot feature in the GNU toolchain.

  - sys-libs/glibc is Prefixified, bug 473728
  - a bug in sys-apps/portage was fixed for sys-kernel/linux-headers in Prefix, bug 470992
  - sys-devel/gcc is patched to output ELF pointing its interpreter field to the dynamic linker in EPREFIX
  - sys-devel/gcc-config is updated for such purpose
  - sys-devel/binutils is patched to have an option to disable runtime
    
    submitted upstream[1]
  - sys-devel/binutils-config is prefixified

    In the Prefix overlay, sys-devel/binutils-config is a wrapper that do the rpath injection
    
  An old deprecated solution was
  - sys-devel/gcc untouched
  - use sys-devel/binutils-config from Prefix overlay to inject dynamic linker instead of rpath

  The solution is documented in the wiki[2] for archiving purpose.

2.1 Remaining Issue 
--------------------
   
   Guys in toolchain and kernel herd are too busy. A list of
   outstanding pending bugs are 478436(kernel-2.eclass),
   478434(toolchain-bintuils.eclass), 474358(toolchain.eclass),
   473728(sys-libs/glibc).

   Can I accelerate the progress of merging these patches by joining
   to toolchain herd, Luca?
   

2.2 Credits 
------------
   
   Ruud Koolen (redlizard) developed the first solution of Prefix with
   libc, and discuss/debated with me back and forth to pin down the
   final solution. Of course the debate will go on as we refine it,
   and especially after he joins Gentoo as well (bug 485618).

3 Profiles 
===========
  Lightweight profiles are created, thanks to the similarity of RAP to
  gentoo-x86.

  At the time of writing, the profiles are organized as followed.

.
|-- default
|   `-- linux
|       |-- amd64
|       |   `-- 13.0
|       |       |-- eapi
|       |       `-- rap
|       |           |-- make.defaults
|       |           `-- parent
|       |-- arm
|       |   `-- 13.0
|       |       |-- armv5te
|       |       |   `-- rap
|       |       |       |-- make.defaults
|       |       |       `-- parent
|       |       `-- armv7a
|       |           `-- rap
|       |               |-- make.defaults
|       |               `-- parent
|       `-- x86
|           `-- 13.0
|               `-- rap
|                   |-- make.defaults
|                   `-- parent
|-- eapi
`-- features
    `-- rap
        |-- make.defaults
        |-- packages
        |-- profile.bashrc
        |-- use.force
        `-- use.mask

  amd64, x86, armv5te (Marvell Kirkwood-based NAS), armv7a (for
  Android devices) are supported and tested.

3.1 Credits 
------------

   Richard Farina (zerochaos) suggested to derive RAP profiles from
   default instead of prefix, which lead to this elegant layout.
   

4 GLEP 
=======
  
  A GLEP draft [3] is composed to standardize RAP. The development
  of RAP is encouraged by the council, and the GLEP will be refined by
  a newly formed GLEP team[4].

4.1 Credits 
------------
   
   Duncan helped with the English expressions of the GLEP draft.

5 Bootstrap Script 
===================

  A bootstrap script deriving from bootstrap-prefix.sh[5] is
  developed[6].

  An alternative bootstrap script by redlizard is being heavily
  tested, it is expected to be more elegant and general than the
  current script.

5.1 Credits 
------------
   
   Steven Trogdon and Richard Farina(zerochaos) helped with the early
   testing of the bootstrap script.

6 Stage 3 Tarball 
==================

  An instance of Gentoo RAP armv7a is bootstrapped on a smartphone, and
  distributed by the Gentoo Infrastructure[7], which in turn is
  tested on a tablet[8].

  A installation guide for normal users is composed as well[9].

6.1 Remaining Issue 
--------------------

   Needs a better solution to configure the glibc DNS resolve and
   uid/gid mapping. Generating the config files from the brother
   Android is preferred.

   The Android usually mount user writable partition with /noexec/,
   which prevents Gentoo RAP from running. Therefore at the time of
   writing we only have a solution for rooted devices. However, it
   might be circumvented by creating a dedicated apk.

7 SD card partition 
====================

  A study has been carried out on the optimized partition and
  filesystem strategy for SD card[10], which is the main extendable
  storage media on Android devices.

  Performance of Gentoo is boosted more than 5 times on small
  files[10], for example portage sync and emerging, on SD cards of
  Android devices after such tuning.

7.1 Remaining Issue 
--------------------
   
   The technical details are still not well understood. The present
   tuning is based trial-and-error and educated guesses.
  

8 Cross triplet-prefix a native toolchain 
==========================================

  For the case no compiler is available in the target device, cross
  compiling a native toolchain is needed.

  A related work is to let crossdev support Prefix, bug 384167.

  A workable solution to make a RAP compatible native toolchain for
  the target device by cross compiling is found and documented[11].

8.1 Credits 
------------
   
   Greg Turner gave many insights on crossdev and toolchain of Prefix,
   and shared his patch for crossdev.

9 Prefix Portage Escalation 
============================

  In some cases, the user does have root privilege and still want to
  install a Gentoo Prefix as normal user and give it root privilege
  when necessary, such as setting suid root and setcap some binaries.

  A simple wrapper proves the concept.

  
#!/bin/bash
ESCULATE="sudo"

scriptpath=${BASH_SOURCE[0]}
scriptname=${scriptpath##*/}

IFS=':'

for path in ${PATH}; do
	[[ -x ${path}/${scriptname} ]] || continue
	[[ ${path}/${scriptname} -ef ${scriptpath} ]] && continue

	elog "using ${ESCULATE} to execute ${path}/${scriptname} $@ as root..."
	${ESCULATE} ${path}/${scriptname} $@

        exit 0
done
exit 1

  
10 Future 
==========
  This Google Summer of Code project is only a beginning of a larger
  roadmap[12].

  Gentoo will share the graphics with Android by leveraging the
  technology of Wayland[13] or Mir[14], instead of only staying
  inside a terminal emulator. To cooperation the development effort, a
  Gentoo subproject[15] has been created. Join it if you are
  interested.

11 Final Acknowledgment 
========================
  
  I am very appreciated to my mentor Luca Barbato. I had a very
  pleasant experience with him since the OpenRC project last year. He
  has a style that matches me quite well, namely pointing at the
  correct direction while not interfering with the very details.

  I have learned a lot in the project. The knowledge gained was
  profound, which *transformed* me: last years' OpenRC project opens
  the blackbox of system booting process, while this years' Android
  project unleashed the magic of toolchain.

  Having such deeper understandings of the operating system, I feel
  more confident and powerful than ever.

  This would be my final year of participation in Google Summer of
  Code, as I am going to get my PhD (on particle physics) and no
  longer a student next year. I will be happy to be a mentor
  myself. Actually I am co-mentoring this years' project of OpenRC
  Debian port[16], as an offshoot of last years' OpenRC project.

  Finally, thanks to all the people who have encouraged, commented on
  or critized the project. Enjoy this, and we will make the universe a
  better place to live =)

All the best,
Benda

[1] [http://article.gmane.org/gmane.comp.gnu.binutils/63137]


[2] [https://wiki.gentoo.org/wiki/Prefix/libc]


[3] [http://git.heroxbd.z.tuna.tsinghua.edu.cn/?p=doc.git;a=blob;f=glep-gap.rst]


[4] [https://www.gentoo.org/proj/en/council/meeting-logs/20130917-summary.txt]


[5] [http://overlays.gentoo.org/proj/alt/browser/trunk/prefix-overlay/scripts/bootstrap-prefix.sh]


[6] [http://dev.gentoo.org/~heroxbd/bootstrap-rap.sh]


[7] [http://distfiles.gentoo.org/experimental/prefix/rap/rap-stage3-armv7a_hardfp-20130924.tar.bz2]


[8] [http://wiki.gentoo.org/wiki/Project:Android/devices]


[9] [http://wiki.gentoo.org/wiki/Project:Android/tarball]


[10] [http://wiki.gentoo.org/wiki/SDCard]


[11] [http://wiki.gentoo.org/wiki/Project:Android/build]


[12] [http://thread.gmane.org/gmane.linux.gentoo.devel/87475]


[13] [http://wayland.freedesktop.org]


[14] [https://wiki.ubuntu.com/Mir]


[15] [http://wiki.gentoo.org/wiki/Project:Android]


[16] [http://anonscm.debian.org/gitweb/?p=collab-maint/openrc.git]