A high-quality suite of tests is crucial in ensuring correctness and robustness of the codebase. Here, we provide instructions how to run unit tests, and also how to add a new one.
Contents
Add your test under the directory tests/python/ or tests/python-gpu/ (if you are testing GPU code). Refer to the PyTest tutorial to learn how to write tests for Python code.
You may try running your test by following instructions in this section.
Add your test under the directory tests/cpp/. Refer to this excellent tutorial on using Google Test.
You may try running your test by following instructions in this section. Note. Google Test version 1.8.1 or later is required.
The JVM packages for XGBoost (XGBoost4J / XGBoost4J-Spark) use the Maven Standard Directory Layout. Specifically, the tests for the JVM packages are located in the following locations:
To write a test for Java code, see JUnit 5 tutorial. To write a test for Scala, see Scalatest tutorial.
You may try running your test by following instructions in this section.
Add your test under the directory R-package/tests/testthat. Refer to this excellent tutorial on testthat.
You may try running your test by following instructions in this section.
Run
make Rcheck
at the root of the project directory.
As part of the building process, tests are run:
mvn package
To run Python unit tests, first install pytest package:
pip3 install pytest
Then compile XGBoost according to instructions in Building the Shared Library. Finally, invoke pytest at the project root directory:
# Tell Python where to find XGBoost module
export PYTHONPATH=./python-package
pytest -v -s --fulltrace tests/python
In addition, to test CUDA code, run:
# Tell Python where to find XGBoost module
export PYTHONPATH=./python-package
pytest -v -s --fulltrace tests/python-gpu
(For this step, you should have compiled XGBoost with CUDA enabled.)
To build and run C++ unit tests enable tests while running CMake:
mkdir build
cd build
cmake -DGOOGLE_TEST=ON -DUSE_DMLC_GTEST=ON ..
make
make test
To enable tests for CUDA code, add -DUSE_CUDA=ON and -DUSE_NCCL=ON (CUDA toolkit required):
mkdir build
cd build
cmake -DGOOGLE_TEST=ON -DUSE_DMLC_GTEST=ON -DUSE_CUDA=ON -DUSE_NCCL=ON ..
make
make test
One can also run all unit test using ctest tool which provides higher flexibility. For example:
ctest --verbose
By default, sanitizers are bundled in GCC and Clang/LLVM. One can enable sanitizers with GCC >= 4.8 or LLVM >= 3.1, But some distributions might package sanitizers separately. Here is a list of supported sanitizers with corresponding library names:
Address sanitizer: libasan
Undefined sanitizer: libubsan
Leak sanitizer: liblsan
Thread sanitizer: libtsan
Memory sanitizer is exclusive to LLVM, hence not supported in XGBoost. With latest compilers like gcc-9, when sanitizer flags are specified, the compiler driver should be able to link the runtime libraries automatically.
One can build XGBoost with sanitizer support by specifying -DUSE_SANITIZER=ON. By default, address sanitizer and leak sanitizer are used when you turn the USE_SANITIZER flag on. You can always change the default by providing a semicolon separated list of sanitizers to ENABLED_SANITIZERS. Note that thread sanitizer is not compatible with the other two sanitizers.
cmake -DUSE_SANITIZER=ON -DENABLED_SANITIZERS="address;leak" /path/to/xgboost
By default, CMake will search regular system paths for sanitizers, you can also supply a specified SANITIZER_PATH.
cmake -DUSE_SANITIZER=ON -DENABLED_SANITIZERS="address;leak" \
-DSANITIZER_PATH=/path/to/sanitizers /path/to/xgboost
Runing XGBoost on CUDA with address sanitizer (asan) will raise memory error. To use asan with CUDA correctly, you need to configure asan via ASAN_OPTIONS environment variable:
ASAN_OPTIONS=protect_shadow_gap=0 ${BUILD_DIR}/testxgboost
By default undefined sanitizer doesn’t print out the backtrace. You can enable it by exporting environment variable:
UBSAN_OPTIONS=print_stacktrace=1 ${BUILD_DIR}/testxgboost
For details, please consult official documentation for sanitizers.