Turnpike Planning Horizons for a Markovian Decision Model

Abstract

This paper establishes some asymptotic properties of the finite state and action space Markovian decision model. For the discounted case, a turnpike theorem is proven which states that an optimal immediate decision when the planning horizon is sufficiently large is to choose one of the decisions which is optimal when the planning horizon is infinite. An upper bound on how large is sufficiently large is given. An asymptotic property of the model for discount factors in a neighborhood of one is also developed.