For my RL model I’ve decided to use a crypto total market cap index, excluding BTC and ETH, as my main source of data, and will probably generate rewards based on that. All well and good.
So I’m also adding the SPY index, but have problems relating to dates and times. I downloaded data from TradingView, and all the dates have the time portion at 13:30 or maybe 14:30. I’m guessing this is the opening time of the US exchanges (probably NYSE), in UTC time. Also, no Saturday or Sunday prices. I’m sure there’s a proper way to convert the datetime to 00:00:00 hours so that I can concatenate the columns with the cryptocap data but I used a hack. Converting the datetime to just the date (with date() function), saving the dataframe to a CSV file, then loading that back in to a new dataframe, parsing the date column, gave me 2024-08-12 00:00:00 instead of the original 2024-08-12 13:30:00. Also, using the asfreq(‘d’) function on the dataframe filled in the missing weekend days, forward filling with Friday prices. All a bit of a hack but I can’t think of any better way to deal with this issue. Not all data I might want to use for this model will be conveniently in daily format, at UTC time.