Month: June 2024

I Wish

TL:DR More whingeing

I’ve mentioned a few times issues related to learning new material, but the one thing I really miss is the graduated example/exercise. Most authors/tutors go from an explanation of the basics to a quite challenging example. In my own teaching I would introduce an easy example/exercise, then a more complex one, and generally a series where the learning at each step was quite doable.

However, I was teaching semester long courses. Four or five hours a week for about sixteen weeks, and had the opportunity to develop a subject in this way. Although my students still considered me to be a lousy teacher, judging by survey results (or non-attendance at my classes).

I don’t think I’ve encountered any learning resources that do what I wish they would do. I guess a good textbook, designed for use in a semester course, would fill the bill. But with most (all?) colleges putting their learning content online (generally in their private learning portal) textbooks are certainly going out of favour.

This post is motivated by my latest difficulty. I’m attempting to make a simplified version of the example provided in the course I’m currently doing. In order to do that I actually need to understand exactly how the complex example works, because I’m getting a bit stuck on how information flows around the system once its up and running. But that somewhat obviates the need for the simpler version, which is to use as a stepping stone. Once again, I’m just going to have to bite the bullet and tackle the harder task head on.

There’s a book on heuristics called How to Solve It by George Polya. One of his strategies when dealing with a complex problem is to solve a simpler problem first. I’ve always found that to be sound advice. I guess working out how someone else’s code works is not exactly solving a problem.

ETA: I’ve found a few simpler problems on Medium. Not the same problem I’m currently tackling, but enough to give me the feel for how information flows around such systems, which is the crucial understanding in my opinion. 🙂

State

From the 6 hour close prices of my synthetic data I have constructed state (a set of features) consisting of (in each row) the period return (fractional change from the previous period), the past 6 periods returns, and the total return over the past 7, 15, 30, 60, 90 and 120 days. Also what day of the week each row occurred on. This can sometimes be significant in trading. Code and dataframe are shown below:

So what is this data used for? Well, the ‘agent’ (in this case the DDQN) gets each row of data and has to work out whether to buy, hold or sell. If it buys then later sells, the profit (or loss) acts as a reward. It can also short sell, i.e. sell first and then buy later. With only this information, and initially acting completely at random, it learns how to make a profit!! Hopefully. Pretty neat, huh?

Synthetic Data

A couple of courses I’ve done have recommended testing an algorithm on synthetic data, especially data with a very simple form such as a fairly linear uptrend or a sine wave (to emulate a mean reverting asset). Each of these should be a bit noisy to be more ‘realistic’ but still, simple. This will show whether the algorithm can learn such simple patterns, and provide the opportunity to test out whether other factors are OK.

One of those ‘other factors’ is amount of data. In the course I’m currently looking at the data used is about 10 years worth of 5 minute data! And when they backtested the algorithm (a double deep Q network) it took about a year to become profitable. That’s a lot of data. Another factor is the state. The course used the whole OHLCV data, with so many lags, at different time granularities, with associated TA indicators, that it ended up with about 160 inputs. My current intention is to simplify, starting with how the input state is organized. But perhaps it’s a good idea to test it out on something easy, such as a noisy uptrend or a noisy sine wave, rather than go straight for real data.

So I made some synthetic data, 6hourly samples of a noisy uptrend, 8000 samples in total. Will my DDQN (Double Deep Q Network) be able to solve this problem?

Plan B

Or is that plan Z? Anyway, several months ago I purchased a course on Quantra about Reinforcement Learning in Trading. I found it pretty heavy going, especially as some of the explanations seemed a little ‘light on’. Well, I’ve just been going through it again (videos, text and code in the form of Jupyter Notebooks) and it makes a lot more sense now that I’ve filled in the details from other sources.

So my plan now is to recreate the template that it develops but in a simpler form. I can always add complexity later. I’ll be using PyTorch instead of Keras/Tensorflow as well, but that change should be pretty trivial now that I’m more familiar with both.

I plan to use ADA as the asset, having already settled on that for my previous plan (development of a ML assisted momentum strategy). Once again I doubt that I’ll actually use this in trading, but it’s a field I’m familiar with so I can focus on setting up the Double Deep Q Network instead of concerning myself with how that relates to the task. I feel pretty confident that I can get something up and running, with lots of scope for improving it after that, and lots of opportunity to test it out on real world data once its working. Definitely seems like a plan.

Biting the Bullet

If I’m going to put time and effort into these studies, it needs to be worthwhile. At the moment the only aspect of Machine Learning that I consider to be worth studying, in terms of its intrinsic interest and possible applications, is Reinforcement Learning. Everything else is just rehashing stuff I already know, at least in general if not in detail.

The basics of RL are pretty boring and pretty difficult. I don’t expect to be posting much for a while as I really need to get some tedious stuff under my belt. Perhaps I’ll resurface on the other side.

ETA: I’ve found a new course that I can relate to a lot better than previous ones. I’m not sure if it’s better teaching, or whether the partial understanding that I’ve achieved from several other books/courses has reached some critical mass. Probably a bit of both. Anyway, I’m enjoying it this time around.

ADA

Started a newish project, implementing the approach from my course using Cardano (ADA) as the asset. Focus at the moment is using XGBoost ML algorithm to help determine whether to go long or short using a momentum strategy. I haven’t used XGBoost before, so something to learn I guess. If the strategy seems profitable, as all the strategies I’ve lost money on have, I might hazard a few digital dollars just to keep the interest going. So here’s the code to fetch the data (ADA-USD) from Yahoo Finance:

import yfinance as yf
import pandas as pd

ada_data = yf.download("ADA-USD", start="2017-01-01",  end="2024-06-02")
ada_data.index = pd.to_datetime(ada_data.index)
print(ada_data)

And here’s the beginning and end of the data received

                Open      High       Low     Close  Adj Close     Volume
Date                                                                    
2017-11-09  0.025160  0.035060  0.025006  0.032053   0.032053   18716200
2017-11-10  0.032219  0.033348  0.026451  0.027119   0.027119    6766780
2017-11-11  0.026891  0.029659  0.025684  0.027437   0.027437    5532220
2017-11-12  0.027480  0.027952  0.022591  0.023977   0.023977    7280250
2017-11-13  0.024364  0.026300  0.023495  0.025808   0.025808    4419440
...              ...       ...       ...       ...        ...        ...
2024-05-28  0.467963  0.468437  0.453115  0.456990   0.456990  418594476
2024-05-29  0.456990  0.463107  0.450914  0.450995   0.450995  350482630
2024-05-30  0.450995  0.454546  0.443807  0.446581   0.446581  356151973
2024-05-31  0.446581  0.454957  0.444461  0.447461   0.447461  290913148
2024-06-01  0.447461  0.452584  0.445254  0.449975   0.449975  167918462

[2397 rows x 6 columns]

I guess I didn’t need that Adjusted Close column, as crypto doesn’t really do dividends and splits the way equities so. Earliest date on Yahoo Finance seems to be 2017-11-09. I wonder if that really was the date Cardano went live. Or perhaps there wasn’t a USD trading pair available.

Next up: Target

I’ve Been Thinking

A few years ago I decided to learn the Python programming language. Before I retired I knew and taught a few other languages but had never learned Python. I did have a book that I read a few pages of but the coding style was so different that I didn’t pursue it. I got really annoyed when the IDE complained about wrong number of indents. Just give me the braces anytime.

After I retired my motivation for learning Python was a strong interest in trading crypto, and the fact that quantitative strategies seemed a lot more sensible to one who lacks any kind of intuition. It became clear that the best language for this sort of thing is Python. So I bit the bullet and these days incorrect indentation is rarely a problem. I put a lot of work into it, a lot of time and quite a bit of money taking courses, buying books, etc. It kept me pretty busy.

Anyone familiar with my recent posts will know that I have become very disenchanted with trading crypto, except the the extremely simple buy and hold strategy for BTC. This doesn’t engage much of my attention, even if I do have the BTC/USDT price chart on my computer most of the time I’m awake.

However, I get bored, very bored. Having a bit of money on the line does wonders for ‘engagement’. Studying ML, for instance, with no actual intention to use it, is just not cutting it. There was a time in my life when I spent a lot of time learning about things with no expectation of any application or reward. Just interest. But that doesn’t seem to work for me anymore. I guess nothing is really new anymore, just variations on familiar themes. I like the big ideas, and implementation details don’t really interest me. Probably why I was never good at languages at school. Learning vocab was just boring, and I had no desire to actually communicate with anyone in, say, French, which I studied for five years in secondary school. Or Latin, which I studied for six.

Anyway, I’m thinking of doing a bit of trading again, not to make any money but just to keep boredom at bay. My grandparents (on my father’s side) used to bet on the horses. I doubt they made any profit doing that. They were working class, there were no books in the house, this was before television, I guess it helped pass the time.

The Bigger They Are…

The new course is working on a momentum strategy, and naturally covers the long/short portfolio. Here one takes a group of similar assets, goes long on the ones that have performed the best recently, and short on the ones that have performed the worst.

Trouble is, things change. The last time I tried that (using returns over the past year as the measure of performance), there was a downturn in the market. All the best performers crashed a lot more than the poorer performers did. I guess they had further to fall. I lost quite a bit of money with that strategy. Not sure that enhancing it with ML will help. I’ve already tried clustering algorithms on crypto assets, without much success. I’ve used K-Means and DBScan. This new course is recommending Heirarchical Clustering. I guess I’ll give ti a try. Probably won’t actually commit any funds to the result though. Not sure what it will take to change my mind on that.

Not So Neat

I’m editing a Jupyter Notebook that came with my latest course to use BTC/USD instead of SPY (S&P500). Not only is this more relevant to my interests but it helps me examine the code closely. Still, the screen can get pretty crowded. Good thing I’ve got a 4K monitor. I discovered that I can get BTC price data back 10 years through one call on yfinance (Yahoo Finance API). Only daily data though.

I need to be mindful of differences between crypto trading and equities trading. For instance a stock exchange typically trades about 252 days per year (Monday to Friday, excluding public holidays) whereas a crypto exchange trades 365.25 days per year. This is important when annualizing metrics such as Sharpe Ratio. In the case of the code shown it’s reflected in the rolling one year return value.

Addict/Idiot

What an idiot I am, or is it just an addict. I’ve purchased another course on trading, quite an expensive one. Having abandoned Statistical Arbitrage (a mean reversion strategy) after about four years of losing money I’m now looking at the other main option, a momentum strategy. I did actually wait 24 hours before purchasing this new course. At least this shows some restraint!

This course is about using ML to assist with a momentum strategy, so it actually has some appeal other than the trading aspect. I hope it’s interesting at least. Will I actually do any trading? Who knows. Now that trading on margin is so much more difficult than it was, probably not. I guess it’s just a way to keep boredom at bay. I have started playing some music (recorder) again, but can’t do that all day.