Sequence Diagram

I think I’ve done this correctly, both from a UML perspective and how my app should actually work. I don’t think I need to create frameworks and such, because the only variations I expect to have is to use different processing algorithms. That would involve using different classes to implement the Processor in the diagram above, and nothing else needs to change. Well, I could change what data I’m working with, but that would only require that I pass a filename to the Environment constructor to tell it what data to load. Easy.

One thing I haven’t included is the possibility of running through the data several times. This would involve having the run method (at No. 6 in the diagram) itself being in a loop. This application is different from most of the book implementations in that it’s basically a continuous process, not episodic, so there isn’t really a terminal state. I guess I could consider running out of capital a terminal state, but I don’t think that’s useful at this stage.

I’ve been reading a more theoretical book on the subject, Reinforcement Learning: An Introduction by Richard Sutton and Andrew Barto. It doesn’t have actual code in it but it does use pseudocode to define various algorithms, and gives a more detailed discussion of all the ins and outs of the subject than the books that focus more on implementation in Python do. I think I’ll be re-reading it quite a few times.