Out-of-sample data

Out-of-sample data is the slice of history you hold back during development. The strategy never sees it while you build and tune, so you can use it to test whether the strategy generalizes to data it had no chance to be fitted to.

This is the test that actually matters. A strategy that looks great on in-sample data but falls apart out-of-sample was overfit to the past. Use the out-of-sample set sparingly: the moment you tweak a strategy in response to its out-of-sample results, that data has effectively become in-sample, and you have lost your clean test.