Regression Testing of Enlightenment

Enlightenment has a regression testing suite that can be of benefit when exploring a possible defect or unexpected change in the window manager or it's applications.

The test suite can be found at https://git.enlightenment.org/devs/zmike/test-e.git but it must be installed using the following instructions before it can be used.

Prerequisites

An up to date installation of EFL from git or from your operating system repository.
The latest enlightenment from our git repository.

Installing

The test suite should be cloned from git and installed in the usual autotools manner:

$ git clone https://git.enlightenment.org/devs/zmike/test-e.git
$ cd test-e
$ ./autogen.sh --prefix=
$ make
$ make install

Running Tests

Tests are invoked using the run script:

./run.sh runs all available tests
./run.sh <test1> <test2> run the specified tests

Graphical output is rendered to a png file matching the name of the test i.e. test1.png.

Test stdout and stderr are saved to a log file with the same name i.e. test1.log.

Configuration

The following environment variables are supported to control the behaviour of the tests:

VG_ON: Run test(s) under valgrind
GDB_ON: Automatically print a backtrace on crash
TEST_EXIT: Use exit() instead of logging out after tests
CLEAN: Run ‘git clean -dxf’ before running tests
DELAY_ON: Wait 4 extra seconds before beginning tests
STRACE_ON: Run tests under strace
E_COMP_ENGINE: ‘sw’ or ‘gl’ to set software or gl rendering
E_WL_FORCE: ‘buffer’, ‘drm’, ‘x11’, ‘wl’ to set the output engine

Additionally you can change the expected outcome of a test by editing the imgs/testname.png file. If you delete the file then it will be recreated from the output of the next test run.

Adding New Tests

To add a new test to the suite follow these steps

Create src/<newtestname>.c
Copy scripts/basic.sh to scripts/<newtestname>.sh
Add src/<newtestname>.c to src/Makefile.mk
Add RUN_TEST(<newtestname>); to src/e_mod_main.c
Add TEST_FN(<newtestname>); to src/e_mod_main.h

Add the following content to src/<newtestname>.c as a starting point:

#include "e_mod_main.h"
TEST_FN(<testname>)
{
    // Initialize stuff here
}