I’ve restructured the RL trading app the way I like it, and sorted out the obvious bugs to the point where it runs without error. Of course there might still be logic errors in the code, and the performance is not great. There seems to be a lot of reshaping, squeezing and unsqueezing tensors along the way, probably more than is actually necessary. I’m going to have to examine parts of the code in more detail. Anyway, running it on my 6hr data for ADAUSDT produced the following plot
Each of the 50 episodes was once through the entire dataset. Probably some serious overfitting there, although no obvious learning took place. However the average reward (return from trade) was greater than zero most of the time.
So writing this the way I want, and fixing all the errors, has improved my understanding of how it all works quite a bit, and put me in a position to explore different variations. I’m feeling pretty happy with progress. I’ve been working on this for several months now.
I added another layer to the neural network. There seem to be fewer results below zero, and not so far below zero. Probably a better result.
