Transient analysis of cumulative measures of markov model behavior

Abstract

Markov chains and Markov reward models provide are useful for modeling fault-tolerant, distributed and multi-processor systems. In this paper, we consider the transient analysis of “cumulative” or “integral” measures of Markov and Markov reward model behavior. These measure include “interval availability” and “expected accumulated reward” over a finite horizon. We consider two methods for numerical model evaluation: Uniformization and differential equation solution. We use a numerical experiment to compare the algorithms' performance as a function of model size, accuracy, and stiffness. Contrary to “folk wisdom”, we observe that cumulative measure solver behavior is usually similar to that seen in instantaneous measure analysis. However, for large time values, cumulative measures do not converge to steady-state values, leading to numerical difficulties like overflow and slow convergence. These problems can be avoided by directly solving time-averaged equations