Browse/search for people

Publication - Professor Sean Collins

    An analysis of transient Markov decision processes


    James, H & Collins, E, 2006, ‘An analysis of transient Markov decision processes’. Journal of Applied Probability, vol 43 (3)., pp. 603 - 621


    A class of Markov decision process is considered in which the boundedness of expected future costs is ensured by a natural form of termination, at least under some policies. Previous treatments of such problems have generally restricted attention to the case where the set of states is finite. In this paper, it is shown that all the results of the finite-state case hold when the set of states is a general Borel space, provided one makes the additional assumption that the optimal value function is bounded below. A sufficient condition is also given for the optimal value function to be bounded below which holds in particular if the set of states is countable.

    Full details in the University publications repository