For the input to the RL model that I’ll use to ‘supervise’ the xgboost model I’ve gathered a crypto market cap index, SPY, Nasdaq, and a crypto Fear and Greed index which goes back to Feb 2018. The others go back further than that but I might have to use the Feb 18 as the start date. This should be enough to make a start on constructing state for the RL model and generating rewards.
My plan at the moment is simply to use the 1 day future return on the cryptomarketcap as the reward. I think I need episodes for the A2C model, so I guess I can just use a set number of days for an episode. I’m using daily data for this, not 6 hourly, which I think will work OK, just have to see. There won’t be any direct interaction between the RL model and the XGBoost model. The process will be that the RL model will predict whether it’s ‘a good time to trade’, i.e. positive return on the current day’s state, or not. If not, then whatever the XGBoost model says about trading ADA specifically will be ignored. If both are in agreement on green to trade, then do so. As the RL model is looking at broader market conditions (SPY, Nasdaq, broad crypto index) then the two models should complement each other, or at least that’s the plan. I think this qualifies as an implementation of meta-labelling, although a very minimal and rough one. Well, the proof is in the pudding, as they say.
Actually, having done a quick review of meta-labelling I’m not sure that what I’m doing does qualify as such. However I did read somewhere some pundit opine that using a deep ML model to ‘critique’ a shallower ML model was a good procedure, and I can certainly make the RL model ‘deeper’ than the XGBoost model, if that actually helps.