There are two types of testing implemented in pyro: unit tests and
regression tests. Both of these are driven by the
script in the root directory.
pyro implements unit tests using
pytest. These can be run via:
pytest -v --nbval pyro
The main driver,
pyro_sim.py has the ability to create benchmarks and
compare output to stored benchmarks at the end of a simulation.
Benchmark output is stored in each solver’s
When testing, we compare zone-by-zone for each variable to see if we
agree exactly. If there is any disagreement, this means that we’ve
made a change to the code that we need to understand (it may be a bug
or may be a fix or optimization).
We can compare to the stored benchmarks simply by running:
When running on a new machine, it is possible that roundoff-level differences may mean that we do not pass the regression tests. In this case, one would need to create a new set of benchmarks for that machine and use those for future tests.