Running compiled code on OSv

This document explains how to run compiled code (C code, C++ code, etc.) on OSv

Compiling Linux applications for OSv

OSv is ABI-compatible with Linux, meaning that it should be able to run (minus any bugs still in OSv...) executable code compiled for Linux, provided this code does not use one of a few features not supported by OSv - such as fork(). This means that you can compile your existing Linux application with its normal build process, and run the resulting Linux executable on OSv.

There's one snag, though: OSv cannot currently run "normal" (fixed-position) executables, and can only run a relocatable shared-object (a file normally given a ".so" extension). It looks for the main() function in that shared object, and runs it.

Converting a compilation process to produce a shared-object instead of an executable is fairly trivial: All you need to do is to add -fPIC option to the compilation of each source file (to produce an object file with position-independent code) and to add -shared to the linking stage, to produce a shared-object instead of an executable. That's it, and you can run the result on OSv (the next section explains how to do that).

By the way, some projects already create ".so"s that can be run in OSv unmodified. For example, the Java runtime (the JVM) is mostly a collection of shared objects, with the "java" executable being a simple main() that calls the right functions from libjvm.so. So to run Java in OSv, all we needed to do is to take unmodified ".so"s (we can take precompiled versions, e.g., from some Linux distribution or from Oracle), and add to it our own simple main which we compiled as explained above (see java/java.cc in OSv's source repository).

Putting the application into OSv's image and running it

There are many ways of getting your application into OSv so you can run it. You can upload the application into a running OSv with scp, or with a Web application. But here we'll describe a more direct way (that is probably not recommended to anyone but OSv developers) - add the executable to OSv's image during OSv's compilation.

During the build of OSv ("make"), it consults a file bootfs.manifest on which files to copy into the image. As an example, we compiled the "Memcached" application to a shared-object memcached.so as described above, and then added it to the built image by adding the following lines to bootfs.manifest:

/memcached.so: /home/nyh/memcached/memcached.so
/libevent-2.0.so.5: /usr/lib64/libevent-2.0.so.5.1.6

Note we needed to copy not only memcached.so, but also libevent.so, a system shared-library that memcached was compiled with, and is needed at run time. As shown, we can simply copy an unmodified shared library (64-bit version, of course) straight out of any Linux distribution, and didn't have to compile libevent ourselves. We put these files in the root directory of the image; If you want to put them in /usr, you'll need to add these lines to usr.manifest instead of bootfs.manifest.

We needed to copy libevent.so, but not the standard C and C++ libraries (libc, libm, libpthread, librt, libdl, libstdc++). Those are already included in the OSv kernel.

Now, to run this new memcached.so, all you need to do is

scripts/run.py -e "memcached.so ..."

For example, to run memcached on a single vCPU, run

scripts/run.py -e "memcached.so -u root -t 1"

Compiling applications which uses OSv APIs

While initially most applications running on OSv will be Linux applications, OSv is not just a Linux clone, and it has additional non-Linux APIs that applications can use to further improve their performance on OSv.

Currently, our build process does not copy OSv's header files anywhere (we'll need to do this in the future). So currently the easiest way to compile a program which uses both Linux and OSv APIs is to to just add this program to OSv's makefile ("build.mk") and have it compile together with OSv with all the right include paths. As an example, consider a version of Memcached modified to use some OSv APIs. We put the source code into a subdirectory "tools/memcached" of OSv's source tree, and add to build.mk:

memcachedobjs := assoc.o cache.o daemon.o hash.o items.o memcached.o \
    slabs.o stats.o thread.o util.o
memcachedobjs := $(addprefix tools/memcached/, $(memcachedobjs))
tools/memcached/memcached.so: CFLAGS+=-levent
tools/memcached/memcached.so: $(memcachedobjs)
        $(makedir)
        $(q-build-so)
$(memcachedobjs): CFLAGS += -fPIC -DHAVE_CONFIG_H -isystem /usr/include
tools += tools/memcached/memcached.so

Now "make", which also builds the "tools" target, will build tools/memcached/memcached.so, and it can be installed on the image as explained above.

The "-isystem /usr/include" is needed above to allow the memcached to also include system header files - namely those of libevent. Without this option, only OSv header files are available, and those provide the standard libc header files, but not additional header files which might be installed on the system (in this example, /usr/include/event.h).