Skip to content

Device Tests & Coverage Report

kai-morich edited this page Feb 18, 2024 · 13 revisions

Only a subset of this library can be tested reasonably with unit tests. Mocking the serial converter chips would hardly represent their real behavior, so I decided to use tests on real devices where the device is connected to:

The instrumented test uses the RFC-2217 server to control the other end of the serial line over Wi-Fi.

Android Setup

By default the USB port is used for ADB debugging, but ADB can also use Wi-Fi. To enable Wi-Fi you have to enable it in Android Settings -> Developer options, connect with USB ADB and enable Wi-Fi with adb tcpip 5555, see more details here.

Serial host setup

You need python + pyserial 3.4 which includes a rfc2217_server.py. You can install pyserial with pip install serial

On Linux the rfc2217_server.py is typically available in usr/share/doc/python-serial/examples. On Windows you have to take it from github.

On a Raspi 3 you have to configure the PL011-UART instead of the mini-UART for the GPIO pins, by using disable-bt or miniuart-bt overlay.

Before executing the test, start the rfc2217 server with: python rfc2217_server.py -v com-port

To see all data apply this patch and use -v -v option.

Serial device setup

Over the time I collected various serial breakout boards, but each device comes with a different pin assignment, so I created my own and soldered mini adapters vor every device.

My pin assignment is:

1 2 3 4 5
GND TxD Rxd n.c. +5V

+5V is connected at all serial devices to provide power to other devices that are not self powered.

The Raspi needs a level shifter because it is not 5V tolerant. I used a 74HC21 which is not the most commonly used level shifter, but that's what I found over the weekend in my parts collection. +5V is not connected as the Raspi as it is self powered.

The Arduino Pro Micro has to be programmed with arduino_leonardo_bridge.ino

Test execution

  • Start rfc2217_server.py on your Windows/Linux host
  • Configure the ip address of your rfc2217 server host in usbSerialForAndroid/build.gradle -> testInstrumentationRunnerArguments -> rfc2217_server_host. Depending on the host type enable/disable the other testInstrumentationRunnerArguments.
  • Connect your device
  • Start DeviceTest
  • When requested, confirm USB permission dialog on your Android device

Tests are sometimes flaky, as they depend on real hardware, which can have varying performance or some data left in receive buffers from previous failing tests. Retrying multiple times, re-attaching the serial device or restarting the rfc2217_server.py typically helps. Else try increasing the timeouts in DeviceTest.java.

Coverage report

Enable apply from: 'coverage.gradle' in usbSerialForAndroid/build.gradle.

Remove previous test results with gradle clean or Android Studio : Menu -> Build -> Clean Project

Run unit tests

Run usb-serial-for-android -> usbSerialForAndroid -> verification -> testAnyDeviceDebugUnitTest

Run device tests

For each device flavor defined in coverage.gradle execute the corresponding gradle task
usb-serial-for-android -> usbSerialForAndroid -> verification -> createDeviceDebugCoverageReport or
usb-serial-for-android -> usbSerialForAndroid -> other -> createDeviceDebugAndroidTestCoverageReport.
Both tasks create the required *.ec and *.xml report files but html reports do not contain annotated source code.

If the tasks are not shown, you may have to enable full task creation with File -> Settings -> Experimental -> Configure all Gradle tasks during Gradle Sync.

Test reports are created even if tests fail. Nevertheless you should check that it didn't fail completely.

Create coverage report

Create summary usbSerialForAndroid/build/reports/jacoco/jacocoTestReport/jacocoTestReport.xml report with gradle task usb-serial-for-android -> usbSerialForAndroid -> other -> jacocoTestReport.

This also creates a html folder with annotated source code.

Upload coverage report

Upload coverage reports to codecov.io using the uploader program. On Windows execute in command prompt (cmd.exe), e.g.

cd c:\work\AndroidStudioProjects\usb-serial-for-android
..\codecov.exe -t <codecov-token>