The Randomization Technique as a Modeling Tool and Solution Procedure for Transient Markov Processes

Abstract

We present a randomization procedure for computing transient solutions to discrete state space, continuous time Markov processes. This procedure computes transient state probabilities. It is based on a construction relating a continuous time Markov process to a discrete time Markov chain. Modifications and extensions of the randomization method allow for computation of distributions of first passage times and sojourn times in Markov processes, and also the computation of expected cumulative occupancy times and expected number of events occurring during a time interval. Several implementations of the randomization procedure are discussed. In particular we present an implementation for a general class of Markov processes that can be described in terms of state space (S), event set (E), rate vectors (R), and target vectors (T)—abbreviated as SERT. This general approach can handle systems whose state spaces are quite large, if they have sparse generators.