Skip to content

Snapshots

For non-textual data structures you can use the snapshot fixture instead of the regtest fixtures.

This function depends on the data type of the snapshot.check argument and pytest-regtest ships with implementations for:

  • Python objects, such as dictionaries or lists
  • NumPy arrays of arbitrary dimensions.
  • pandas data frames.
  • polars data frames.

The user can provide own handlers for particular data structures.

Write a snapshot test for a nested Python data structure

For non-textual data structures you can use the snapshot fixture instead of the regtest fixture. This fixture offers a method .check:

# test_snapshot.py

def test_nested_data(snapshot):
    li = list(range(5))
    text = "lorem ipsum " * 8
    data = dict(li=li, text=text)
    snapshot.check(data)

Run the test

If you run this test script with pytest the first time there is no recorded output for this test function so far and thus the test will fail with a message including a diff:

$ pytest -v test_snapshot.py
============================= test session starts ==============================
platform linux -- Python 3.12.12, pytest-9.0.3, pluggy-1.6.0 -- /home/docs/checkouts/readthedocs.org/user_builds/pytest-regtest/checkouts/latest/.venv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/tmpuerscz_r
plugins: regtest-2.5.0, cov-7.1.0
collecting ... collected 1 item

test_snapshot.py::test_nested_data FAILED                                [100%]

=================================== FAILURES ===================================
_______________________________ test_nested_data _______________________________

snapshot error(s) for test_snapshot.py::test_nested_data:

snapshot not recorded yet:
    > test_snapshot.py +5
    > snapshot.check(data)
    {'li': [0, 1, 2, 3, 4],
     'text': 'lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem '
             'ipsum lorem ipsum lorem ipsum '}
---------------------------- pytest-regtest report -----------------------------
total number of failed regression tests: 0
total number of failed snapshot tests  : 1
=========================== short test summary info ============================
FAILED test_snapshot.py::test_nested_data
============================== 1 failed in 0.12s ===============================

This is a diff of the current output is to a previously recorded output tobe. Since we did not record output yet, the diff contains no lines marked +.

Reset the test

To record the current output, we run pytest with the --regtest-reset flag:

$ pytest -v --regtest-reset test_snapshot.py
============================= test session starts ==============================
platform linux -- Python 3.12.12, pytest-9.0.3, pluggy-1.6.0 -- /home/docs/checkouts/readthedocs.org/user_builds/pytest-regtest/checkouts/latest/.venv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/tmpuerscz_r
plugins: regtest-2.5.0, cov-7.1.0
collecting ... collected 1 item

test_snapshot.py::test_nested_data RESET                                 [100%]

---------------------------- pytest-regtest report -----------------------------
total number of failed regression tests: 0
total number of failed snapshot tests  : 0
the following output files have been reset:
  _regtest_outputs/test_snapshot.test_nested_data__0
============================== 1 passed in 0.01s ===============================

You can also see from the output that the recorded output is in the _regtest_outputs folder which in the same folder as the test script. Don't forget to commit this folder to your version control system!

Run the test again

$ pytest -v test_snapshot.py
============================= test session starts ==============================
platform linux -- Python 3.12.12, pytest-9.0.3, pluggy-1.6.0 -- /home/docs/checkouts/readthedocs.org/user_builds/pytest-regtest/checkouts/latest/.venv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/tmpuerscz_r
plugins: regtest-2.5.0, cov-7.1.0
collecting ... collected 1 item

test_snapshot.py::test_nested_data PASSED                                [100%]

---------------------------- pytest-regtest report -----------------------------
total number of failed regression tests: 0
total number of failed snapshot tests  : 0
============================== 1 passed in 0.01s ===============================

Break the test

Let us break the test by changing the length of li from 5 to 6 and the number of repetitions when constructing text from 8 to 7:

# test_snapshot.py

def test_nested_data(snapshot):
    li = list(range(6))
    dd = dict(li=li)
    text = "lorem ipsum " * 7
    data = dict(li=li, text=text)
    snapshot.check(data)

The next run of pytest delivers a nice diff of the current and expected output from this test function:

$ pytest -v test_snapshot.py
============================= test session starts ==============================
platform linux -- Python 3.12.12, pytest-9.0.3, pluggy-1.6.0 -- /home/docs/checkouts/readthedocs.org/user_builds/pytest-regtest/checkouts/latest/.venv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/tmpuerscz_r
plugins: regtest-2.5.0, cov-7.1.0
collecting ... collected 1 item

test_snapshot.py::test_nested_data FAILED                                [100%]

=================================== FAILURES ===================================
_______________________________ test_nested_data _______________________________

snapshot error(s) for test_snapshot.py::test_nested_data:

snapshot mismatch:
    > test_snapshot.py +6:
    > snapshot.check(data)
    --- current
    +++ expected
    @@ -1,3 +1,3 @@
    -{'li': [0, 1, 2, 3, 4, 5],
    +{'li': [0, 1, 2, 3, 4],
      'text': 'lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem '
    -         'ipsum lorem ipsum '}
    +         'ipsum lorem ipsum lorem ipsum '}
---------------------------- pytest-regtest report -----------------------------
total number of failed regression tests: 0
total number of failed snapshot tests  : 1
=========================== short test summary info ============================
FAILED test_snapshot.py::test_nested_data
============================== 1 failed in 0.12s ===============================

In case the change was intended, you can reset the test again:

$ pytest -v --regtest-reset test_snapshot.py
============================= test session starts ==============================
platform linux -- Python 3.12.12, pytest-9.0.3, pluggy-1.6.0 -- /home/docs/checkouts/readthedocs.org/user_builds/pytest-regtest/checkouts/latest/.venv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/tmpuerscz_r
plugins: regtest-2.5.0, cov-7.1.0
collecting ... collected 1 item

test_snapshot.py::test_nested_data RESET                                 [100%]

---------------------------- pytest-regtest report -----------------------------
total number of failed regression tests: 0
total number of failed snapshot tests  : 0
the following output files have been reset:
  _regtest_outputs/test_snapshot.test_nested_data__0
============================== 1 passed in 0.01s ===============================

Multiple snapshots in one test

If you call snapshot.check multiple times, pytest-regtest uses the call index to distinguish files. For better readability, you can use the version argument:

def test_multi_snapshots(snapshot):
    snapshot.check({"id": 1}, version="header")
    # ... some processing ...
    snapshot.check({"data": [1, 2, 3]}, version="payload")

This will create two separate output files in the _regtest_outputs folder.

Using version is especially useful when snapshot.check is called in a loop. The failure report includes the file path and line number of the snapshot.check call, which is the same for every iteration — so without version you cannot tell which iteration failed. Passing a meaningful version makes each iteration distinguishable in the report:

def test_dump_tables(snapshot):
    for table_name in ("users", "orders", "products"):
        snapshot.check(dump_table(table_name), version=table_name)

A failure on the "orders" iteration is then reported with version='orders', and the recorded file is named after the table name too.

Platform specific snapshots

The version argument can also be used to implement platform-specific snapshots, similar to how it is used for the regtest fixture. For details and examples, please refer to the Platform specific output section in the regtest documentation.

JSON format for Python object snapshots

By default, Python object snapshots are stored as binary pickle files (object.pkl). You can switch to a human-readable JSON format by passing format="json" to snapshot.check():

def test_nested_data(snapshot):
    data = {"li": list(range(5)), "text": "lorem ipsum " * 8}
    snapshot.check(data, format="json")

Unlike the binary pickle format, JSON files can be opened in any text editor and are easy to review: you can read exactly what was recorded, spot regressions at a glance, and get meaningful diffs in pull requests.

Trade-off: JSON only works for JSON-serializable objects (dicts, lists, strings, numbers, booleans, None). For arbitrary Python types (e.g. custom classes, numpy arrays) you must keep the default "pickle" format.

Migrating existing snapshots: change format="pickle" (or omit it) to format="json" and run pytest --regtest-reset. The old object.pkl file is removed and a new object.json file is written. When loading, pytest-regtest tries object.pkl first and falls back to object.json, so existing tests keep working until you reset them.

Custom failure handling

You can provide a failure_handler callback that is executed only when the snapshot comparison fails:

def on_failure(current, recorded):
    print(f"Custom log: expected {recorded}, got {current}")

def test_with_custom_log(snapshot):
    # we use a helper object to show its usage:
    my_obj = {"a": 1}
    snapshot.check(my_obj, failure_handler=on_failure)