Swatinem Blog Resume

Please delete your Snapshot Tests

— 4 min

Snapshot testing is quite popular. And unfortunately most of the time it is the wrong tool for the job.

In software projects, testing is extremely important. A piece of software can only be as good as its testsuite, at least when it is changing over time.

# Tests, abstractly

So what is the purpose of tests in the first place? Well, you test that for certain inputs, your program returns some outputs or performs some side effects.

For each test case you should be able to very specifically say what it is asserting. For snapshot tests, the answer to the question “what is your test asserting” is most often “hm… I guess… everything?”

Maybe when you created the snapshot test, you had an idea of what exactly you wanted to assert, but over time this is being lost.

Another reason snapshot tests are bad is because they change way too often. Completely unrelated changes in the codebase can change your snapshot output. Because over time you lose track of what you actually wanted to assert with the test, a snapshot that frequently changes will just be accepted by both the developer and the reviewer. The test assertions thus lose their purpose.

# Good assertions don’t change

Ideally, each test should assert something very specific. A ground truth that you know is true and is set in stone for infinity. Good test assertions should never change. Sure, your test code will change as your API evolves. But the assertions should not.

If a test assertion changes, it means either the assertion was bad from the start, or you have a regression somewhere that needs to be investigated.

Snapshot tests change way too frequently, and they are way too broad which means people get into the habit of “yeah, whatever”.

As mentioned before, you should assert very specific outcomes and side effects. Snapshot tests frequently assert intermediate artifacts which are not interesting and change a lot.

# Good snapshot tests

I have only seen a few snapshot tests that were done right, and it very much depends on the software under test if snapshot tests make sense or not.

As all other kind of tests, the testcases should be as minimal as possible. They should just test one specific case, and not the whole world.

The cases where I think snapshot testing makes sense if you have transformations on text.

I do snapshot testing in rollup-plugin-dts. I test very specific use-cases with each testcase. I test end to end, asserting the final output as a snapshot. These tests are stable and should never change when I make changes to the core logic. They do change however in the very rare case that my main dependency rollup changes some of its logic.

Another example of where snapshot testing makes sense would be rust-analyzer. You give it a snippet of code and a cursor position. Then you apply a suggestion and you assert the final output.

Snapshotting intermediate artifacts like abstract syntax trees in this example would be bad as those can change frequently without influencing the final output.

# General testing advice

I would like to end this post with some general recommendation towards testing: