Analysis of an Important Class of Non-Markov Systems

Abstract

Probabilistic modeling of many types of systems generally assumes Markov behavior. However, some important practical systems exhibit memory. For example, in digital computer systems, the probability of occurrence of a transient failure is related to the time period the system has been operating correctly. Analytic methods do not yet exist that allow accurate modeling of such systems for the purpose of reliability analysis and fault-tolerant design. Methods are presented here to analyze an important class of non-Markov systems. In this class, the transition-probability-rate of an out-ward transition from a state is related to the duration the system has continuously been in that state. To analyze such systems, concept of memory profile has been introduced. Methods are first presented which enable computation of steady-state probabilities for both discrete-time and continuous-time processes with two states. These are then extended for general non-steady-state cases and also for systems with more than two states.