Month: October 2024

For a few months I’ve been anticipating the publication of Reinforcement Learning for Finance by Yves Hilpisch, supposedly in December, according to Amazon. However I found today that it is already available, and have purchased the Kindle edition. All the books I have on RL deal with non-financial problems, except for one by Ivan Gridin that has a chapter on RL in stock pricing, although it is a bit simplistic. So this text promises to fill a gap, which is exactly what Dr. Hilpisch said in his introduction.

I’m just about to get into it, and fortunately I have become a lot more comfortable with Tensorflow lately, and I expect that’s what he’ll be using. I don’t know how much time I’ll be able to devote to this at the moment – I have a few pressing matters to sort out over the next couple of months. However there’s no way I can not get started, at least. I’m hoping that all the study that I’ve already done on RL will make this book fairly easy to work through. Anyway, we’ll see.

Update:

Well, that was disappointing. I’ve reached Chapter 2, which starts to discuss DQNs, and the associated code gives me errors. The (first) error message suggested a fix, but in trying to implement it I ran into dependency issues and now I don’t know where I stand. I should be able to run it on the remote platform provided by the company associated with the author, but having bought one book there I have difficulty registering a second or third. I should eventually be able to work on their REPL platform, but it might take a while. The book was only published a week ago. I guess I can get something out of it without running their code. I don’t know Tensorflow well enough to be able to hack problems like this (yet).

On further thought:

One reason for using Tensorflow is the large amount of learning material available which uses that library. However it seems the change from Keras 2 to Keras 3 was sufficiently large to break older code. I’m rethinking my decision. Besides, it’s the principles that I need to learn, not any specific implementation. Of course getting code up and running helps learning principles, so a bit of a chicken and egg situation. I just need to master one library sufficiently that I can implement anything in it. I’m leaning towards PyTorch.

Discretionary trading (aka trading on intuition) is what I’m doing at the moment, trading the ETH/BTC pair, short only. I can’t borrow and trade on Binance, but I can borrow elsewhere (AAVE), transfer to Binance, then trade it for BTC, so effectively a short even though Binance does not allow me to margin trade completely on Binance. I’m only trading a small amount, basically paper trading except that I am using a small amount rather than ‘fake’ coins. I’ll see what results I get in six to twelve months. Ernie Chan says anything less than that is not a meaningful test of a strategy.

I do check the XGBoost predictions from time to time, but there’s a bit of a disconnect between the predictions and my intuition. I’ll work on refining the signals when I get a bit more time. At the moment I’m taking a break from machine learning. A while ago I mentioned implementing a neural network that uses LSTM, and that’s certainly on my todo list. Preparing to move house is taking time and energy at the moment. PLus I’m getting out and getting some exercise, rather than sitting at the computer all day.

I’ve been concentrating on machine learning for quite a while now, but before that I had a significant interest in visual art. I’m getting back into photography after a long hiatus. For many years I’ve been doing computer graphics, both 2D and 3D but I started with photography. If interested check out my other blog.

I finally got an LSTM up and running, courtesy of the course Neural Networks in Trading from Quantinsti. So, just running their code, but it really isn’t much more complicated than an ordinary MLP. Also I’m warming to Tensorflow. It has some nice features such as saving the current best model as training progresses, and it appears to integrate well with scikit-learn, which has some pipeline related features that could be useful once things get more complex.

The above graph represents an interesting idea. The spread between predictions and actual value is plotted and trades using a mean reversion strategy. Bollinger Bands at 2 STD are shown. So many possibilities. I could include some version of an LSTM in an ensemble along with my XGBoost model and the (furure) RL model. Maybe I’ll find something that is consistently profitable some day after all.