An issue frequently discussed in Machine Learning is look ahead bias, or training a model which has access to data from beyond the training period. I plan to use a training set for the backtest, and a test set for validation, rather than training (backtesting in this case) on all available data and then validating by paper trading (a form of forward testing).
So look ahead bias could be introduced by checking for cointegration on the whole data series. If I’m backtesting on, say, three out of four years of data, checking for cointegration on all the data involves using data that did not exist during the training data set. Any statistics calculated, even just a mean or std, will be calculated on data that should not have been available.
So, to be rigorous, I should probably split my data into the training period (I’ll probably use 2020 – 2022), a test period (2023 – current) and save them as separate files so there is no possibility of introducing look ahead bias. If backtesting on the training period produces a strategy, or more importantly the parameters for the strategy, since I’ll be using mean reversion, that continues to work with the following year + of data, then perhaps it will continue to work a while longer.
With all the ‘cointegration’ that I’ve encountered that turns out not to be, it raises the question of whether there are any pairs that exhibit persistent stationarity, and not just the ephemeral variety. By using only mature coins (ADA, EOS, LTC, etc) and testing over long periods, and then checking for ongoing stationarity over further relatively long periods I hope to find a few pairs that are a little more robust than most of what I’ve encountered so far. No doubt there will be very few such pairs, so diversification will not be an option, but clearly something has to give.
The coins I’m starting with are ADA, ALGO, ATOM, EOS, ETC and LTC. I’ll probably expand this list after initial testing, but hopefully I’ll find something out of that batch. I’ve decided to use 6hr data beginning Jan 1st 2020. Something to get started with.