bionic/libc/docs/OVERVIEW.TXT

Bionic C Library Overview:
==========================

Introduction:

Core Philosophy:

  The core idea behind Bionic's design is: KEEP IT REALLY SIMPLE.

  This implies that the C library should only provide lightweight wrappers
  around kernel facilities and not try to be too smart to deal with edge cases.

  The name "Bionic" comes from the fact that it is part-BSD and part-Linux:
  its source code consists in a mix of BSD C library pieces with custom
  Linux-specific bits used to deal with threads, processes, signals and a few
  others things.

  All original BSD pieces carry the BSD copyright disclaimer. Bionic-specific 
  bits carry the Android Open Source Project copyright disclaimer. And
  everything is released under the BSD license.

Architectures:

  Bionic currently supports the ARM and x86 instruction sets. In theory, it
  should be possible to support more, but this may require a little work (e.g.
  adding system call IDs to SYSCALLS.TXT, described below, or modifying the
  dynamic linker).

  The ARM-specific code is under arch-arm/ and the x86-specific one is under
  arch-x86/

  Note that the x86 version is only meant to run on an x86 Android device. We
  make absolutely no claim that you could build and use Bionic on a stock x86
  Linux distribution (though that would be cool, so patches are welcomed :-))

Syscall stubs:

  Each system call function is implemented by a tiny assembler source fragment
  (called a "syscall stub"), which is generated automatically by
  tools/gensyscalls.py which reads the SYSCALLS.TXT file for input.

  SYSCALLS.TXT contains the list of all syscall stubs to generate, along with
  the corresponding syscall numeric identifier (which may differ between ARM
  and x86), and its signature

  If you modify this file, you may want to use tools/checksyscalls.py which
  checks its content against official Linux kernel header files, and will
  report errors when invalid syscall ids are used.

  Sometimes, the C library function is really a wrapper that calls the
  corresponding syscall with another name. For example, the exit() function
  is provided by the C library and calls the _exit() syscall stub.

  See SYSCALLS.TXT for documentation and details.


time_t:

  time_t is 32-bit as defined by the kernel on 32-bit CPUs. A 64-bit version
  would be preferrable to avoid the Y2038 bug, but the kernel maintainers
  consider that this is not needed at the moment.

  Instead, Bionic provides a <time64.h> header that defines a time64_t type,
  and related functions like mktime64(), localtime64(), etc...


Timezone management:

  The name of the current timezone is taken from the TZ environment variable,
  if defined. Otherwise, the system property named 'persist.sys.timezone' is
  checked instead.

  The zoneinfo timezone database and index files are located under directory
  /system/usr/share/zoneinfo, instead of the more Posix-compliant path of
  /usr/share/zoneinfo


off_t:

  For similar reasons, off_t is 32-bit. We define loff_t as the 64-bit variant
  due to BSD inheritance, but off64_t should be available as a typedef to ease
  porting of current Linux-specific code.


Linux kernel headers:

  Bionic comes with its own set of "clean" Linux kernel headers to allow
  user-space code to use kernel-specific declarations (e.g. IOCTLs, structure
  declarations, constants, etc...). They are located in:

     ./kernel/common,
     ./kernel/arch-arm
     ./kernel/arch-x86

  These headers have been generated by a tool (kernel/tools/update-all.py) to
  only include the public definitions from the original Linux kernel headers.

  If you want to know why and how this is done, read kernel/README.TXT to get
  all the (gory) details.


PThread implementation:

   Bionic's C library comes with its own pthread implementation bundled in.
   This is different from other historical C libraries which:

    - place it in an external library (-lpthread)
    - play linker tricks with weak symbols at dynamic link time

   The support for real-time features (a.k.a. -lrt) is also bundled in the
   C library.

   The implementation is based on futexes and strives to provide *very* short
   code paths for common operations. Notable features are the following:

      - pthread_mutex_t, pthread_cond_t are only 4 bytes each.

      - Normal, recursive and error-check mutexes are supported, and the code
        path is heavily optimized for the normal case, which is used most of
        the time.

      - Process-shared mutexes and condition variables are not supported.
        Their implementation requires far more complexity and was absolutely
        not needed for Android (which uses other inter-process synchronization
        capabilities).

        Note that they could be added in the future without breaking the ABI
        by specifying more sophisticated code paths (which may make the common
        paths slightly slower though).

      - There is currently no support for read/write locks, priority-ceiling in
        mutexes and other more advanced features. Again, the main idea being
        that this was not needed for Android at all but could be added in the
        future.

pthread_cancel():

   pthread_cancel() will *not* be supported in Bionic, because doing this would
   involve making the C library significantly bigger for very little benefit.

   Consider that:

     - A proper implementation must insert pthread cancellation checks in a lot
       of different places of the C library. And conformance is very difficult
       to test properly.

     - A proper implementation must also clean up resources, like releasing
       memory, or unlocking mutexes, properly if the cancellation happens in a
       complex function (e.g. inside gethostbyname() or fprintf() + complex
       formatting rules). This tends to slow down the path of many functions.

     - pthread cancellation cannot stop all threads: e.g. it can't do anything
       against an infinite loop

     - pthread cancellation itself has short-comings and isn't very portable
       (see http://advogato.org/person/slamb/diary.html?start=49 for example).

   All of this is contrary to the Bionic design goals. If your code depends on
   thread cancellation, please consider alternatives.

   Note however that Bionic does implement pthread_cleanup_push() and
   pthread_cleanup_pop(), which can be used to handle cleanups that happen when
   a thread voluntarily exits through pthread_exit() or returning from its
   main function.


pthread_once():

  Do not call fork() within a callback provided to pthread_once(). Doing this
  may result in a deadlock in the child process the next time it calls
  pthread_once().

  Also, you can't throw a C++ Exception from the callback (see C++ Exception
  Support below).

  The current implementation of pthread_once() lacks the necessary support of
  multi-core-safe double-checked-locking (read and write barriers).


Thread-specific data

  The thread-specific storage only provides for a bit less than 64
  pthread_key_t objects to each process. The implementation provides 64 real
  slots but also uses about 5 of them (exact number may depend on
  implementation) for its own use (e.g. two slots are pre-allocated by the C
  library to speed-up the Android OpenGL sub-system).

  Note that Posix mandates a minimum of 128 slots, but we do not claim to be
  Posix-compliant.

  Except for the main thread, the TLS area is stored at the top of the stack.
  See comments in bionic/libc/bionic/pthread.c for details.

  At the moment, thread-local storage defined through the __thread compiler
  keyword is not supported by the Bionic C library and dynamic linker.


Multi-core support

  At the moment, Bionic does not provide or use read/write memory barriers.
  This means that using it on certain multi-core systems might not be
  supported, depending on its exact CPU architecture.


Android-specific features:

  Bionic provides a small number of Android-specific features to its clients:

  - access to system properties:

       Android provides a simple shared value/key space to all processes on the
       system. It stores a liberal number of 'properties', each of them being a
       simple size-limited string that can be associated to a size-limited
       string value.

       The header <sys/system_properties.h> can be used to read system
       properties and also defines the maximum size of keys and values.

   - Android-specific user/group management:

       There is no /etc/passwd or /etc/groups in Android. By design, it is
       meant to be used by a single handset user. On the other hand, Android
       uses the Linux user/group management features extensively to secure
       process permissions, like access to various filesystem directories.

       In the Android scheme, each installed application gets its own
       uid_t/gid_t starting from 10000; lower numerical ids are reserved for
       system daemons.

       getpwnam() recognizes some hard-coded subsystems names (e.g. "radio")
       and will translate them to their low-user-id values. It also recognizes
       "app_1234" as the synthetic name of the application that was installed
       with uid 10000 + 1234, which is 11234. getgrnam() works similarly

       getgrouplist() will always return a single group for any user name,
       which is the one passed as an input parameter.

       getgrgid() will similarly only return a structure that contains a
       single-element members list, corresponding to the user with the same
       numerical value than the group.

       See bionic/libc/bionic/stubs.c for more details.

    - getservent()

       There is no /etc/services on Android. Instead the C library embeds a
       constant list of services in its executable, which is parsed on demand
       by the various functions that depend on it. See
       bionic/libc/netbsd/net/getservent.c and
       bionic/libc/netbsd/net/services.h

       The list of services defined internally might change liberally in the
       future. This feature is mostly historically and is very rarely used.

       The getservent() returns thread-local data. getservbyport() and
       getservbyname() are also implemented in a similar fashion.

     - getprotoent()

       There is no /etc/protocol on Android. Bionic does not currently
       implement getprotoent() and related functions. If added, it will
       likely be done in a way similar to getservent()

DNS resolver:

  Bionic uses a NetBSD-derived resolver library which has been modified in
  the following ways:

     - don't implement the name-server-switch feature (a.k.a. <nsswitch.h>)

     - read /system/etc/resolv.conf instead of /etc/resolv.conf

     - read the list of servers from system properties. the code looks for
       'net.dns1', 'net.dns2', etc.. Each property should contain the IP
       address of a DNS server.

       these properties are set/modified by other parts of the Android system
       (e.g. the dhcpd daemon).

       the implementation also supports per-process DNS server list, using the
       properties 'net.dns1.<pid>', 'net.dns2.<pid>', etc... Where <pid> stands
       for the numerical ID of the current process.

     - when performing a query, use a properly randomized Query ID (instead of
       a incremented one), for increased security.

     - when performing a query, bind the local client socket to a random port
       for increased security.

     - get rid of *many* unfortunate thread-safety issues in the original code

  Bionic does *not* expose implementation details of its DNS resolver; the
  content of <arpa/nameser.h> is intentionally blank. The resolver
  implementation might change completely in the future.


PThread Real-Time Timers:

  timer_create(), timer_gettime(), timer_settime() and timer_getoverrun() are
  supported.

  Bionic also now supports SIGEV_THREAD real-time timers (see timer_create()).
  The implementation simply uses a single thread per timer, unlike GLibc which
  uses complex heuristics to try to use the less threads possible when several
  timers with compatible properties are used.

  This means that if your code uses a lot of SIGEV_THREAD timers, your program
  may consume a lot of memory. However, if your program needs many of these
  timers, it'd better handle timeout events directly instead.

  Other timers (e.g. SIGEV_SIGNAL) are handled by the kernel and use much less
  system resources.


Binary Compatibility:

  Bionic is *not* in any way binary-compatible with the GNU C Library, ucLibc
  or any known Linux C library. This means several things:

  - You cannot expect to build something against the GNU C Library headers and
    have it dynamically link properly to Bionic later.

  - You should *really* use the Android toolchain to build your program against
    Bionic. The toolchain deals with many important details that are crucial
    to get something working properly.

  Failure to do so will usually result in the inability to run or link your
  program, or even runtime crashes. Several random web pages on the Internet
  describe how you can succesfully write a "hello-world" program with the
  ARM GNU toolchain. These examples usually work by chance, if anything else,
  and you should not follow these instructions unless you want to waste a lot
  of your time in the process.

  Note however that you *can* generate a binary that is built against the
  GNU C Library headers and then statically linked to it. The corresponding
  executable should be able to run (if it doesn't use dlopen()/dlsym())


Dynamic Linker:

  Bionic comes with its own dynamic linker (just like ld.so on Linux really
  comes from GLibc). This linker does not support all the relocations
  generated by other GCC ARM toolchains.


C++ Exceptions Support:

  At the moment, Bionic doesn't support C++ exceptions, what this really means
  is the following:

    - If pthread_once() is called with a C++ callback that throws an exception,
      then the C library will keep the corresponding pthread_once_t mutex
      locked. Any further call to pthread_once() will result in a deadlock.

      A proper implementation should be able to register a C++ exception
      cleanup handler before the callback to properly unlock the
      pthread_once_t. Unfortunately this requires tricky assembly code that
      is highly dependent on the compiler.

      This feature is not planned to be supported anytime soon.

    - The same problem may arise if you throw an exception within a callback
      called from the C library. Fortunately, these cases are very rare in the
      real-world, but any callback you provide to the C library should *not*
      throw an exception.

    - Bionic lacks a few support functions to have exception support work
      properly.

System V IPCs:

  Bionic intentionally does not provide support for System-V IPCs mechanisms,
  like the ones provided by semget(), shmget(), msgget(). The reason for this
  is to avoid denial-of-service. For a detailed rationale about this, please
  read the file docs/SYSV-IPCS.TXT.

Include Paths:

  The Android build system should automatically provide the necessary include
  paths required to build against the C library headers. However, if you want
  to do that yourself, you will need to add:

      libc/arch-$ARCH/include
      libc/include
      libc/kernel/common
      libc/kernel/arch-$ARCH

  to your C include path.