Continuous Integration

We have two CI systems:

Wrench

This is an internal CI system, where the main build occurs, packages are produced and all tests executed: Wrench.

A full build + test run typically takes ~5 hours.

Jenkins

We have a Jenkins job that builds and tests all pull requests: Jenkins PR builder.

In order to make development more agile and get a quicker response to pull requests (the goal is take maximum 1 hour per build), support for device and some tests are disabled.

The default set of tests are:

• api tests (apitest for XM and monotouch-test for XI).
• various linker tests (dont link, link sdk, link all).
• introspection tests (for both XI and XM).
• msbuild tests (for both XI and XM).

This can be changed by setting various labels on the pull requests.

• enable-device-build: This will enable device support. This will also run more tests (all tests in the default set of tests that are ignored because they require device support).
• run-*-tests: This will enable the specified tests.
• skip-*-tests: This will disable the specified tests.

The enable-device-build label must be set immediately after creating the pull request, because it must be set before the build starts on the Jenkins bot (a couple of minutes after creating the pull request if there are idle bots). The *-tests labels do not have to be set until the tests start executing (it should take at least 20 minutes to build first).

If using hub from the command line to create pull requests, you can pass --labels=<comma separated list of labels> to add the labels when creating the pull request.

Some of the otherwise disabled tests will also be automatically enabled according to which files are modified in a pull request.