Test in openspdm

Besides SpdmEmu and UnitTest introduced in readme, openspdm also supports some other tests.

Prerequisit

Build Tool

cmake for Windows and Linux.

Run Test

Test other ARCH (ARM, AArch64, RiscV32, RiscV64, ARC)

Linux support only.

Install compiler:

sudo apt-get install gcc-arm-linux-gnueabi
sudo apt-get install gcc-aarch64-linux-gnu
sudo apt-get install gcc-riscv64-linux-gnu

Build RiscV32 compiler:

sudo apt-get install autoconf automake autotools-dev curl python3 libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev
git clone --recursive https://github.com/riscv/riscv-gnu-toolchain
cd riscv-gnu-toolchain
./configure --prefix=/opt/riscv32 --with-arch=rv32gc --with-abi=ilp32d
sudo make linux
sudo ln -s /opt/riscv32/bin/* /usr/bin

Build ARC compiler:

sudo apt-get install -y texinfo byacc flex libncurses5-dev zlib1g-dev libexpat1-dev texlive build-essential git wget gawk bison xz-utils make python3 rsync locales
mkdir arc_gnu
cd arc_gnu
git clone https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain.git
git clone https://github.com/foss-for-synopsys-dwc-arc-processors/binutils-gdb.git binutils
git clone https://github.com/foss-for-synopsys-dwc-arc-processors/gcc.git
git clone --reference binutils https://github.com/foss-for-synopsys-dwc-arc-processors/binutils-gdb.git gdb
git clone https://github.com/foss-for-synopsys-dwc-arc-processors/newlib.git
git clone https://github.com/wbx-github/uclibc-ng.git # For For Linux uClibc toolchain
git clone https://github.com/foss-for-synopsys-dwc-arc-processors/glibc.git # For Linux glibc toolchain
git clone https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux

cd toolchain
./build-all.sh --no-elf32 --cpu hs38 --install-dir $INSTALL_ROOT 
# This command will build toolchain for ARC HS Linux development, for other ARC cores please refer to https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/blob/arc-releases/README.md

sudo ln -s /<work_dir>/arc_gnu/toolchain/bin/* /usr/bin

Install qemu.

sudo apt-get install build-essential pkg-config zlib1g-dev libglib2.0-0 libglib2.0-dev  libsdl2-dev libpixman-1-dev libfdt-dev autoconf automake libtool librbd-dev libaio-dev flex bison -y
wget https://download.qemu.org/qemu-4.2.0.tar.xz
tar xvf qemu-4.2.0.tar.xz
cd qemu-4.2.0
./configure --prefix=/usr/local/qemu --audio-drv-list=
sudo make -j 8 && sudo make install
sudo ln -s /usr/local/qemu/bin/* /usr/local/bin

Run test

For ARM: qemu-arm -L /usr/arm-linux-gnueabi <TestBinary>

For AArch64: qemu-aarch64 -L /usr/aarch64-linux-gnu <TestBinary>

For RiscV32: qemu-riscv32 -L /opt/riscv32/sysroot <TestBinary>

For RiscV64: qemu-riscv64 -L /usr/riscv64-linux-gnu <TestBinary>

Collect Code Coverage

Code Coverage in Windows with DynamoRIO

Download and install DynamoRIO 8.0.0. Then set DRIO_PATH=<DynameRIO_PATH>

Install Perl ActivePerl 5.26.

Build cases. Goto openspdm/Build/<TARGET>_<TOOLCHAIN>/<ARCH>. mkdir log and cd log.

Run all tests and generate log file : %DRIO_PATH%\<bin64|bin32>\drrun.exe -c %DRIO_PATH%\tools\<lib64|lib32>\release\drcov.dll -- <test_app>

Generate coverage data with filter : %DRIO_PATH%\tools\<bin64|bin32>\drcov2lcov.exe -dir . -src_filter openspdm

Generate coverage report : perl %DRIO_PATH%\tools\<bin64|bin32>\genhtml coverage.info

The final report is index.html.
Code Coverage in Linux with GCC and lcov.

Install lcov sudo apt-get install lcov.

Build cases. Goto openspdm/Build/<TARGET>_<TOOLCHAIN>/<ARCH>. mkdir log and cd log.

Run all tests.

Collect coverage data : lcov --capture --directory <openspdm_root_dir> --output-file coverage.info

Collect coverage report : genhtml coverage.info --output-directory .

The final report is index.html.

Run Fuzzing

Fuzzing in Linux with AFL

Download and install AFL. Unzip and follow docs\QuickStartGuide.txt. Build it with make. Ensure AFL binary is in PATH environment variable.
```
export AFL_PATH=<AFL_PATH>
export PATH=$PATH:$AFL_PATH
```
Then run commands as root (every time reboot the OS):
```
sudo bash -c 'echo core >/proc/sys/kernel/core_pattern'
cd /sys/devices/system/cpu/
sudo bash -c 'echo performance | tee cpu*/cpufreq/scaling_governor'
```
Known issue: Above command cannot run in Windows Linux Subsystem.

Build cases with AFL toolchain: make Fuzzing -f GNUmakefile ARCH=<X64|Ia32> TARGET=<DEBUG|RELEASE> TOOLCHAIN=AFL CRYPTO=<MbedTls|Openssl> -e WORKSPACE=<openspdm_root_dir>

Run cases:
```
mkdir testcase_dir
mkdir /dev/shm/findings_dir
cp <seed> testcase_dir
afl-fuzz -i testcase_dir -o /dev/shm/findings_dir <test_app> @@
```
Note: /dev/shm is tmpfs.
Fuzzing in Windows with winafl

Clone winafl. Download DynamoRIO.

Set path set AFL_PATH=<AFL_PATH> and set DRIO_PATH=<DynameRIO_PATH>.

NOTE: as known issue googleprojectzero/winafl#145 that cause compatibility issues in recent Windows versions, the author has disabled Drsyms in recent WinAFL builds, if you want you use the newest version, please according to Method.2 to rebuild winafl yourself.

Build winafl:
```
mkdir [build32|build64]
cd [build32|build64]
cmake -G"Visual Studio 16 2019" -A [Win32|x64] .. -DDynamoRIO_DIR=%DRIO_PATH%\cmake -DUSE_DRSYMS=1
cmake --build . --config Release
```
NOTE: If you get errors where the linker couldn't find certain .lib files. please refer to googleprojectzero/winafl#145 and delete the nonexistent files from "Additional Dependencies".

Copy all binary under [build32|build64]/bin/Release to [bin32|bin64]. robocopy /E /is /it [build32|build64]/bin/Release [bin32|bin64].

Build cases with VS2019 toolchain. (non AFL toolchain in Windows): nmake Fuzzing ARCH=<X64|Ia32> TARGET=<DEBUG|RELEASE> TOOLCHAIN=VS2019 CRYPTO=<MbedTls|Openssl> -e WORKSPACE=<openspdm_root_dir>

Run cases:
```
cp <test_app> winafl\<bin64|bin32>
cp <test_app_pdb> winafl\<bin64|bin32>
cd winafl\<bin64|bin32>
afl-fuzz.exe -i in -o out -D %DRIO_PATH%\<bin64|bin32> -t 20000 -- -coverage_module <test_app> -fuzz_iterations 1000 -target_module <test_app> -target_method main -nargs 2 -- <test_app> @@
```
Fuzzing in Linux with LLVM LibFuzzer

Ensure LLVM binary in in PATH environment variable.
```
export CLANG_PATH=<LLVM_PATH>/bin
export ASAN_SYMBOLIZER_PATH=$CLANG_PATH/llvm-symbolizer
```
Build cases with LIBFUZZER toolchain: make Fuzzing -f GNUmakefile ARCH=<X64|Ia32> TARGET=<DEBUG|RELEASE> TOOLCHAIN=LIBFUZZER CRYPTO=<MbedTls|Openssl> -e WORKSPACE=<openspdm_root_dir>

Run cases:
```
mkdir NEW_CORPUS_DIR // Copy test seeds to the folder before run test
<test_app> NEW_CORPUS_DIR -rss_limit_mb=0 -artifact_prefix=<OUTPUT_PATH>
```
Fuzzing in Windows with LLVM LibFuzzer

Note: Please install 64bit exe for X64 build (IA32 build is not supported with LLVM9)

Ensure LLVM binary in in PATH environment variable.
```
set LLVM_PATH=<LLVM_PATH>
set PATH=%PATH%;%LLVM_PATH%\bin
```
Build cases with LIBFUZZER toolchain: nmake Fuzzing ARCH=X64 TARGET=<DEBUG|RELEASE> TOOLCHAIN=LIBFUZZER CRYPTO=<MbedTls|Openssl> -e WORKSPACE=<openspdm_root_dir>

Run cases:
```
mkdir NEW_CORPUS_DIR // Copy test seeds to the folder before run test
<test_app> NEW_CORPUS_DIR -rss_limit_mb=0 -artifact_prefix=<OUTPUT_PATH>
```

Run Symbolic Execution

KLEE

Download and install KLEE with LLVM9. Please follow all 12 steps including optional ones.

In step 3, constrint solver STP is recommended here. Set size of the stack to a very large value: $ ulimit -s unlimited.

In step 8, below example can be use:
```
$ cmake \
   -DENABLE_SOLVER_STP=ON \
   -DENABLE_POSIX_RUNTIME=ON \
   -DENABLE_KLEE_UCLIBC=ON \
   -DKLEE_UCLIBC_PATH=/home/tiano/env/klee-uclibc \
   -DGTEST_SRC_DIR=/home/tiano/env/googletest-release-1.7.0 \
   -DENABLE_UNIT_TESTS=ON \
   -DLLVM_CONFIG_BINARY=/usr/bin/llvm-config \
   -DLLVMCC=/usr/bin/clang \
   -DLLVMCXX=/usr/bin/clang++
   /home/tiano/env/klee
```
Ensure KLEE binary is in PATH environment variable.
```
export KLEE_SRC_PATH=<KLEE_SOURCE_DIR>
export KLEE_BIN_PATH=<KLEE_BUILD_DIR>
export PATH=$KLEE_BIN_PATH:$PATH
```
Build cases in Linux with KLEE toolchain. (KLEE does not support Windows) make -f GNUmakefile ARCH=<X64|Ia32> TARGET=<DEBUG|RELEASE> TOOLCHAIN=KLEE CRYPTO=<MbedTls|Openssl> -e WORKSPACE=<openspdm_root_dir>

Use KLEE to generate ktest: klee --only-output-states-covering-new <test_app>

Transfer .ktest to seed file, which can be used for AFL-fuzzer. python UnitTest/Fuzzing/Tools/TransferKtestToSeed.py <Arguments>

Arguments: the path of .ktest file. ... the paths of .ktest files. the path of folder contains .ktest file. ... the paths of folders contain .ktest file.

Run Model Checker

CBMC

Install CBMC tool. For Windows, unzip cbmc-5-10-win. For Linux, unzip cbmc-5-11-linux-64. Ensure CBMC executable directory is in PATH environment variable.

Build cases with CBMC toolchain:

For Windowns, open visual studio 2019 command prompt at openspdm dir and type nmake ARCH=Ia32 TOOLCHAIN=CBMC TARGET=<DEBUG|RELEASE> CRYPTO=MbedTls -e WORKSPACE=<openspdm_root_dir>. (Use x86 command prompt for ARCH=Ia32 only)

For Linux, open command prompt at openspdm dir and type make -f GNUmakefile ARCH=X64 TOOLCHAIN=CBMC TARGET=<DEBUG|RELEASE> CRYPTO=MbedTls -e WORKSPACE=<openspdm_root_dir>. (ARCH=X64 only)

The output binary is created by the goto-cc.

For more infomration on how to use CBMC, please refer to CBMC Manual, such as properties, modeling-nondeterminism, api. Example below:

Using goto-instrument static analyzer operates on goto-binaries and generate a modified binary: goto-instrument SpdmRequester.exe SpdmRequester.gb <instrumentation-options>

Using CBMC on the modified binary: cbmc SpdmRequester.gb --show-properties

Run Static Analysis

Use Klocwork in windows as an example.

Install Klocwork and set environment.

set KW_HOME=C:\Klocwork
set KW_ROOT=%KW_HOME%\<version>\projects_root
set KW_TABLE_ROOT=%KW_HOME%\Tables
set KW_CONFIG=%KW_ROOT%\projects\workspace\rules\analysis_profile.pconf
set KW_PROJECT_NAME=openspdm

Build openspdm with Klocwork :

kwinject --output %KW_ROOT%\%KW_PROJECT_NAME%.out nmake ARCH=<X64|Ia32> TARGET=<DEBUG|RELEASE> CRYPTO=<MbedTls|Openssl> -e WORKSPACE=<openspdm_root_dir>

Collect analysis data :

kwservice start
kwadmin create-project %KW_PROJECT_NAME%
kwadmin import-config %KW_PROJECT_NAME% %KW_CONFIG%
kwbuildproject --project %KW_PROJECT_NAME% --tables-directory %KW_TABLE_ROOT%\%KW_PROJECT_NAME% %KW_ROOT%\%KW_PROJECT_NAME%.out --force
kwadmin load %KW_PROJECT_NAME% %KW_TABLE_ROOT%\%KW_PROJECT_NAME%

View report at http://localhost:8080/.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test.md

Test.md

Test in openspdm

Prerequisit

Build Tool

Run Test

Test other ARCH (ARM, AArch64, RiscV32, RiscV64, ARC)

Collect Code Coverage

Run Fuzzing

Run Symbolic Execution

Run Model Checker

Run Static Analysis

Files

Test.md

Latest commit

History

Test.md

File metadata and controls

Test in openspdm

Prerequisit

Build Tool

Run Test

Test other ARCH (ARM, AArch64, RiscV32, RiscV64, ARC)

Collect Code Coverage

Run Fuzzing

Run Symbolic Execution

Run Model Checker

Run Static Analysis