My early tests on my RL trading app gave promising results, which at the time I thought were a little ‘too good to be true’. Well, that feeling was justified, as I later discovered that I had written the code in a way that it repeatedly learned from a small subset of the data, thus essentially ‘rote learning’ (called overfitting in ML lingo) rather than learning more general principles that could generalize to unseen data.
So after rewriting the code, and fixing many other errors besides (for which the logging I’ve incorporated has proved somewhat helpful), and also downloading 6 hour data from Binance for the entire 6 years that they’ve had ADAUSDT on their exchange, I’ve been running the app again with a variety of optimizers but always getting a similar result, which is that almost no learning takes place!! My average return over several thousand trades is approx 0.03%. Not enough to even cover the fees (which I haven’t included in the algorithm).
There’s not much point trying to do further optimization, or explore a range of different network topologies, when the baseline is so close to zero. I think I’m going to have to address the ‘what data to use’ issue first up, until I do actually get some learning, and then try to improve on it. That Quantra course used quite a lot of input features, including several technical indicators and what day of the week it was. I’m going to have to enlarge my ‘state space’ a bit. It’s interesting to see what other people (authors of books/courses) are using for their features. Ideally I should be using some measure of market sentiment. Perhaps I need to learn how to scrape X for tweets (?) relating to crypto. Another day, perhaps.