.TH "doc_TESTING_md" 3elektra "Sun May 29 2016" "Version 0.8.14" "Elektra" \" -*- nroff -*-
.ad l
.nh
.SH NAME
doc_TESTING_md \- TESTING 

.SS "Introduction"
.PP
Libraries need a pervasive testing for continuous improvement\&. Any problem found and behaviour described must be written down as test and integrated so that after running all tests in the build directory: 
.PP
.nf
make run_all

.fi
.PP
.PP
and on the target (installed) system: 
.PP
.nf
kdb run_all

.fi
.PP
.PP
assures that all these promises hold\&.
.PP
To run memcheck tests use: 
.PP
.nf
make run_memcheck

.fi
.PP
.PP
In the build directory, internally ctest is used, so you can also call ctest with its options\&. On the target (installed) system our own scripts drive the tests\&.
.PP
.SS "Conventions"
.PP
.IP "\(bu" 2
All names of the test must start with test
.IP "\(bu" 2
No tests should run if ENABLE_TESTING is OFF\&.
.IP "\(bu" 2
All tests that access harddisc:
.IP "  \(bu" 4
should be tagged with kdbtests
.IP "  \(bu" 4
should not shall run, if ENABLE_KDB_TESTING is OFF\&.
.IP "  \(bu" 4
should only write below /tests ans system/mountpoints
.PP

.IP "\(bu" 2
If your test has memleaks, e\&.g\&. because the library used leaks and that cannot be fixed, give them the label memleak with following command:
.PP
set_property(TEST testname PROPERTY LABELS memleak)
.PP
.PP
.SS "Strategy"
.PP
The testing must happen on every level of the software to achieve a maximum coverage with the available time\&. In the rest of the document we describe the different levels and where these tests are\&.
.PP
.SS "CFramework"
.PP
This is basically a bunch of assertion macros and some output facilities\&. It is written in pure C and very lightweight\&.
.PP
It is located \fBhere\fP\&.
.PP
.SS "ABI Tests"
.PP
C ABI Tests are written in plain C with the help of cframework\&.
.PP
The main purpose of these tests are, that the binaries of old versions can be used against new versions as ABI tests\&.
.PP
So lets say we compile Elektra 0\&.8\&.8 (at this version dedicated ABI tests were introduced) in the -full variant\&. But when we run the tests, we use libelektra-full\&.so\&.0\&.8\&.9 (either by installing it or by setting LD_LIBRARY_PATH)\&. You can check with ldd which version is used\&.
.PP
The tests are located \fBhere\fP\&.
.PP
.SS "C Unit Tests"
.PP
C Unit Tests are written in plain C with the help of cframework\&.
.PP
It is used to test internal data structures of libelektra that are not ABI relevant\&.
.PP
ABI tests can be done on theses tests, too\&. But by nature from time to time these tests will fail\&.
.PP
They are located \fBhere\fP\&.
.PP
.SS "Module Tests"
.PP
The modules, which are typically used as plugins in Elektra (but can also be available statically or in the -full variant), should have their own tests\&.
.PP
Use the Cmake macro add_plugintest for adding these tests\&.
.PP
.SS "C++ Unit Tests"
.PP
C++ Unit tests are done using the gtest framework\&. See \fBarchitectural\fPdecision"\&.
.PP
Use the CMake macro add_gtest for adding these tests\&.
.PP
.SS "Script Tests"
.PP
Script test are done using POSIX shell + CMake\&. See \fBarchitectural\fPdecision"\&.
.PP
The script tests have different purposes:
.IP "\(bu" 2
End to End tests (usage of tools as a user would do)
.IP "\(bu" 2
External Compilation tests (compile and run programs as a user would do)
.IP "\(bu" 2
Conventions tests (do internal checks that check for common problems)
.IP "\(bu" 2
Meta Test Suites (run other test suites)
.PP
.PP
.SS "Other kind of Tests"
.PP
Bindings, other than C++ typically have their own way of testing\&.
.PP
.SS "Fuzz Testing"
.PP
copy some import files to testcase_dir and run:
.PP
/usr/src/afl/afl-1\&.46b/\&./afl-fuzz -i testcase_dir -o findings_dir bin/kdb import user/tests