The Bandit Algorithms by Tor Lattimore and Prof. Csaba Szepesv´ari provides an introduction to the multi-armed bandit problem. This includes different approaches for solving this type of problems using stochastic, adversarial, and Bayesian frameworks.
