The Chromium project has an general utility library referred to as libbase. Because it is standalone and does not depend on any other parts of Chromium, it has been been picked up by other Google related projects so people don't have to reinvent these things.
Along those lines, a package in Chromium OS is provided so that projects specific to us can share the code without having to bundle it multiple times over. Currently, there are over 20 such projects in Chromium OS. To keep people on their toes, the package is named libchrome as libbase by itself is too confusing. Granted, libchrome isn‘t exactly clear itself, but it’s a lose-lose situation, and we tried our best. Please still love us.
Note: If your package is integrated into the platform2 ebuild, then this is already handled for you in the common platform2 ebuild and you can skip this section.
There are 4 things to make sure the ebuild does when building a Chromium OS package that uses libchrome:
cros-debug eclasscros-debug-add-NDEBUG in one of src_prepare, src_configure, or src_compile.BASE_VER to the version you are depending onBelow you can find copy & paste snippets that should work for any ebuild in the Chromium OS tree. All you should need to change is the 125070 as the number is updated.
...
inherit cros-debug
...
LIBCHROME_VERS="125070"
...
RDEPEND="chromeos-base/libchrome:${LIBCHROME_VERS}[cros-debug=]"
...
src_compile() {
...
tc-export PKG_CONFIG
cros-debug-add-NDEBUG
export BASE_VER=${LIBCHROME_VERS}
...
}
If your package has been upgraded to platform2 (if not, why not?), then it's simple.
In your package's BUILD.gn, list libchrome as a dependency as needed (e.g. either in the project-common target_defaults or the target-specific section). For example, portier takes the former approach:
pkg_config("target_defaults") {
pkg_deps = [
"libbrillo-${libbase_ver}",
"libchrome-${libbase_ver}",
"protobuf",
]
}
At time of writing, the platform2 build system automatically takes care of the rest for you.
In a standard common.mk Chromium OS platform project, you can use these snippets in your Makefile:
# If the build env has exported $PKG_CONFIG to a wrapper, use that, else use # the default pkg-config wrapper (so we can "make" in place for testing). PKG_CONFIG ?= pkg-config # If the build env has exported $BASE_VER, use that. Else, use the last # version that we tested against. This allows the ebuild to update to a # newer version without having to explicitly update the source build # system for trivial changes. BASE_VER ?= 125070 # You can add as many or as view pkg-config libraries to this PC_DEPS # value. Here we just use a specific version of libbase. PC_DEPS = libchrome-$(BASE_VER) # Look up the compiler flags and linker settings via the pkg-config # wrapper once. That is why we use a dedicated variable and the := # operator -- if we use =, then make will end up executing the # pkg-config wrapper many times. PC_CFLAGS := $(shell $(PKG_CONFIG) --cflags $(PC_DEPS)) PC_LIBS := $(shell $(PKG_CONFIG) --libs $(PC_DEPS))
In your source, include files from base/ like normal. So if you want to use the string printf API, do:
#include <base/stringprintf.h>
Over time, we‘ve evolved how we package up the base source tree. Here we’ll cover the lessons we've learned, and why we do the things that we do. This way, future decisions can take into consideration all the factors without missing something.
For some more background (and specifics), see this thread.
The first iteration involved creating a single libbase.a from a single version of Chromium (and install headers into /usr/include/base/). Then anyone who wanted to use this code base would link with -lbase and #include <base/foo.h>.
The advantages here:
libbase.a archive was actually a bunch of .o files, and only the needed ones were pulled in).Unfortunately, as it was widely deployed, we hit scaling issues:
libbase.a needed third-party libraries, projects had to manually link those in themselves (like -lrt or -lm or -lpthread or -lglib-2.0 or ...).libbase.a changed over time, so many projects ended up just copying and pasting a long list of libraries leading to overlinking (pulling in libraries that at runtime never get used), which in turn leads to runtime overhead.-I paths to many third-party projects just in case libbase needed them.libbase.a and sometimes it wouldn't (depending on the order of libraries during link time!). This is a bad state to be in terms of reproducibility and reliability.The first few points were annoying, but bearable. The latter points (related to upgrading), however, were not. So we embarked on a search for a solution!
Since static libraries were so trivial to throw together, we tried to expand on this concept. Now, rather than installing libbase.a, we would install libbase-<version>.a and /usr/include/base-<version>/base/. Then projects would be built and linked against a specific version of libbase by using e.g. -lbase-85268 and compiled against a matching set of headers by using e.g. -I/usr/include/base-85268.
This has advantages over a single static library:
build_packages still works!) for people.However, after building and testing this prototype, we hit problems that were non-starters for deploying related to libraries. When a Chromium OS shared library (e.g. libmetrics) links against a specific libbase version, it turns around and exports the symbols from that version. So when linking an application against a different version of libbase, and that shared library, we cannot guarantee that the ABI between the two versions are the same. While the compile might have been clean, we don't know if the runtime will be clean, and the behavior could change depending on the link order (-lbase-### -lmetrics vs -lmetrics -lbase-###). This could easily lead to hard-to-debug and hard-to-understand crashes or misbehavior at runtime.
We could say that the shared library would clean up its exported ABI by never exporting any libbase symbols (which would be a good thing regardless), but the resulting system is still unmanageable. Since linking against the static library pulls in both functions and state, and some of the APIs from libbase have initialization functions (such as the logging framework which sets up message headers, files, and maintains a buffer), libmetrics now has its own copy of the logging code, and the end application has its own copy of logging code. When the application initializes the logging framework, it initializes its own logging state, not libmetrics' logging state. So when libmetrics attempts to do logging of its own, it often times will crash as its logging state is uninitialized. We could add initialization functions to libmetrics so that it too would initialize its own libbase state, but now we have two parallel logging functions at runtime which could be clobbering the external state (e.g. files).
Similar complications come up when using static libraries that themselves use libbase. If libmetrics was built against libbase-1234, and another project was built against libbase-5678, they'll want different functions at link time which can lead to symbol clashing leading either to link time failures or runtime failures (as detailed above).
This is all fairly fragile, and is arguably trading one set of problems (hard to build and upgrade) for a different, and perhaps even worse, set of problems (things are easy to build and upgrade, but hard to run and debug). We can do better, so let's think harder.
Since the slotted aspect of the previous proposal got us the upgrade path that we desired, we now just have to solve the duplicate state problem. That could be done by only using shared libraries with libbase -- there's no chance to have multiple states be linked in as there are no static archives anymore. Now instead of providing libbase-<version>.a, we provide libbase-<version>.so. The other aspects of the previous proposal are the same (include paths, as well as compiling and linking flags).
The advantages here:
DT_NEEDED ELF tags).It's not all peaches and apple pie though -- there are some trade-offs here:
libfoo.a which uses libbase and multiple other projects use libfoo.a) as this is hard to track, and leads to similar problems as slotted static libraries in terms of mismatch of ABI.libbase-1234.so and libfoo.so, and libfoo.so will load libbase-5678.so, and both libbases will attempt to satisfy symbol references leading to runtime ABI conflicts and possibly random crashes.The last point here is the only real show stopper. Fortunately, two things work in our favor. Generally, the ABI is stable with libbase (across the version ranges we upgrade between), so the runtime “mostly works”. This means we can tolerate a period of time where we are upgrading to a newer version of libbase but some applications are actually (runtime) linked against multiple versions. By the time we actually release, the upgrade will have completed, so there will once again only be one version live at a time. Further, since we can detect exactly what versions of a library an ELF has been linked against, we can confidently detect the cases where a program uses one version of libbase, but links against a shared library which pulls in a different one and act appropriately (i.e. update all the packages).
At this point, we have a workable solution. But we can still do better. Onwards!
Since we need to change all projects that use libbase (from -lbase to -lbase-<version>, and adding -I/usr/include/base-<version>), we might as well come up with a better answer overall that scales and addresses other annoyances. This brings us to pkg-config.
The advantages of providing a .pc file for projects to query are significant!
-lrt or -lpthread or -I/usr/include/... or anything else, the .pc file declares everything it depends on. Projects then just ask for the CFLAGS and libraries that libbase needs, and it gets expanded as needed..pc file. No end projects need change.For the nit-pickers out there, there are disadvantages:
pkg-config at build time.So this cleans up the compiling/linking process nicely, and integrates with existing pkg-config framework that other libraries depend on.
The biggest disadvantage to shared libraries is that libbase isn‘t really one API, but rather a large collection of different APIs. Some require third party libraries to work (like pcre or glib or pthreads), while others require very little. So forcing one project that wants just the simple APIs (like string functions) to pull in more complicated APIs which pull in other third party libraries (like pcre) even though it won’t use them is a waste of runtime resources.
We can combat this though by leveraging some linker tricks. When you specify a library like -lfoo, it doesn't have to be just a static archive (libfoo.a) or a shared ELF (libfoo.so), it could even be a linker script! Combined with the useful AS_NEEDED directive, we can create an arbitrary number of smaller shared libraries (like libbase-core-1234 and libbase-pcre-1234 and libbase-foo-1234) where each one has its own additional library requirements and provides different sets of APIs. Then when people link against -lbase-1234, the linker will look at all of the smaller libbase shared libraries and only pull in the ones we actually use. This is all transparent to the user of the libbase API.
So we have all of the advantages of slotted dynamic libraries, and only one of the downsides: we still have possible runtime conflicts where a program uses one version, but a library it links against uses a different version. As noted previously, this is an acceptable trade off for now, and makes the upgrade situation significantly more manageable.
Some of libbase's headers define structs or classes that include or exclude members based on whether the NDEBUG macro is defined or not. If libbase is built with NDEBUG defined, but then a program that dynamically links against libbase includes those headers with NDEBUG undefined (or vice versa), resulting in disagreements about the sizes of objects, hard-to-debug segfaults can occur. We define a cros-debug USE flag to try to ensure that NDEBUG is set or unset consistently across different packages.
Here are some random thoughts that might be worth investigating to try and improve the current situation:
See the gn build recipe .
It maintains a list of all the files which go into a shared library fragment (such as ‘core’ and ‘glib’ and ‘event’). It's largely split along the lines of what third party libraries will get pulled in (so the ‘core’ only requires C libraries, ‘glib’ additionally requires glib, ‘event’ additionally requires libevent, etc...). The fragments could conceivably be split further, but the trade-offs in terms of runtime overhead were found to not warrant it (generally in the range of “system noise”).
This build file is also responsible for generating the pkg-config .pc file and linker script.
Once a Chromium OS project has identified newer APIs that they want to utilize, or the upstream Chromium project has introduced features that make it more attractive to us (such as no longer hard requiring GTK+), it's time for us to schedule an upgrade! Here is a general approach which should work for people:
Cr-Original-Commit-Position: refs/heads/master@{#334285}.aosp/master branch (this is the code that‘s built by the libchrome ebuild). This is a new, exciting process -- hope you’re ready for an adventure!Android.mk file for building libchrome on Android, an SConstruct file for building it on Chrome OS, and MODULE_LICENSE_BSD and NOTICE files.libchrome-#### ebuild and update the revision it uses from the AOSP repository.BASE_VER value.platform2-9999.ebuild code to depend on the new libchrome version.LIBCHROME_VERS array and just add the new #### to it.platform.gyp file in src/platform/common-mk/.targets.dependencies section.platform2-9999.ebuild to only depend on the latest libchrome version.platform.gyp to only include latest libchromeos version..gyp files.BASE_VER in libchromeos code base.This really should be done within a single development cycle so that we aren‘t wasting system resources by shipping (and having active) multiple libbase shared libraries. One of the advantages of using shared libraries is that all of the non-writable sections (like .text) get shared between processes, and that’s defeated in part by multiple active libbase versions.
libbase on Google Git