Testing

Testing#

There are two types of testing implemented in pyro: unit tests and regression tests. Both of these are driven by the test.py script in the root directory.

Unit tests#

pyro implements unit tests using pytest. These can be run via:

pytest -v --nbval

Regression tests#

The main driver, pyro_sim.py has the ability to create benchmarks and compare output to stored benchmarks at the end of a simulation. Benchmark output is stored in each solver’s tests/ directory. When testing, we compare zone-by-zone for each variable to see if we agree exactly. If there is any disagreement, this means that we’ve made a change to the code that we need to understand (it may be a bug or may be a fix or optimization).

We can compare to the stored benchmarks simply by running:

./pyro/test.py

Note

When running on a new machine, it is possible that roundoff-level differences may mean that we do not pass the regression tests. In this case, one would need to create a new set of benchmarks for that machine and use those for future tests.