Markovian real-time adaptive control of signal systems