Eager Markov Chains

Parosh Aziz Abdulla, Noomene Ben Henda, Richard Mayr, Sven Sandberg

Research output: Chapter in Book/Report/Conference proceedingConference contribution


We consider infinite-state discrete Markov chains which are eager: the probability of avoiding a defined set of final states for more than n steps is bounded by some exponentially decreasing function f(n). We prove that eager Markov chains include those induced by Probabilistic Lossy Channel Systems, Probabilistic Vector Addition Systems with States, and Noisy Turing Machines, and that the bounding function f(n) can be effectively constructed for them. Furthermore, we study the problem of computing the expected reward (or cost) of runs until reaching the final states, where rewards are assigned to individual runs by computable reward functions. For eager Markov chains, an effective path exploration scheme, based on forward reachability analysis, can be used to approximate the expected reward up-to an arbitrarily small error.
Original languageEnglish
Title of host publicationAutomated Technology for Verification and Analysis
Subtitle of host publication4th International Symposium, ATVA 2006, Beijing, China, October 23-26, 2006.
PublisherSpringer Berlin Heidelberg
Number of pages20
ISBN (Electronic)978-3-540-47238-4
ISBN (Print)978-3-540-47237-7
Publication statusPublished - 2006


Dive into the research topics of 'Eager Markov Chains'. Together they form a unique fingerprint.

Cite this