Finished another chapter, discussing one of the most basic problems in human life. Decisions. Specifically, how to transition from collecting information required to make a decision, to actually implementing that decision.
In ML terms the initial phase is called exploration. In the case of our one armed bandits (see previous post) it involves trying them all out to see which one (if any) gives the best payout. Exploitation is then using that best bandit to make money. Problem is if you spend too much time on exploration you’re spending a lot of money on losing machines. If you spend too little time/money on exploration then the ‘best’ one may be just a fluke, and not profitable in the long term.
A couple of the policies regarding the transition discussed in the book look at a soft transition from exploration to exploitation. During the exploitation phase occasionally recheck the ‘losing’ machines to gather a bit more data to confirm the decision. And if the ‘winning’ machine starts to produce bad results in the long term recheck the others more often. Rather like a bad marriage I guess.