Parity Objectives in Countable MDPs

Richard Mayr; Stefan Kiefer; Mahsa Shirmohammadi; Dominik Wojtczak

doi:10.1109/LICS.2017.8005100

Parity Objectives in Countable MDPs

Richard Mayr, Stefan Kiefer, Mahsa Shirmohammadi, Dominik Wojtczak

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We study countably infinite MDPs with parity objectives, and special cases with a bounded number of colors in the Mostowski hierarchy (including reachability, safety, Buchi and co-Buchi).
In finite MDPs there always exist optimal memoryless deterministic (MD) strategies for parity objectives, but this does not generally hold for countably infinite MDPs. In particular, optimal strategies need not exist. For countable infinite MDPs, we provide a complete picture of the memory requirements of optimal (resp., ϵ-optimal) strategies for all objectives in the Mostowski hierarchy. In particular, there is a strong dichotomy between two different types of objectives. For the first type, optimal strategies, if they exist, can be chosen MD, while for the second type optimal strategies require infinite memory. (I.e., for all objectives in the Mostowski hierarchy, if finite-memory randomized strategies suffice then also MD strategies suffice.) Similarly, some objectives admit ϵ-optimal MD-strategies, while for others ϵ-optimal strategies require infinite memory. Such a dichotomy also holds for the subclass of countably infinite MDPs that are finitely branching, though more objectives admit MD-strategies here.

Original language	English
Title of host publication	2017 32nd Annual ACM/IEEE Symposium on Logic in Computer Science (LICS)
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Number of pages	21
ISBN (Electronic)	978-1-5090-3018-7
DOIs	https://doi.org/10.1109/LICS.2017.8005100
Publication status	Published - 18 Aug 2017
Event	2017 32nd Annual ACM/IEEE Symposium on Logic in Computer Science - Reykjavik, Reykjavik, Iceland Duration: 20 Jun 2017 → 23 Jun 2017 http://lics.siglog.org/lics17/

Conference

Conference	2017 32nd Annual ACM/IEEE Symposium on Logic in Computer Science
Abbreviated title	LICS 2017
Country/Territory	Iceland
City	Reykjavik
Period	20/06/17 → 23/06/17
Internet address	http://lics.siglog.org/lics17/

Keywords

countable MDPs, parity objectives, strategies, memory requirement

Access to Document

10.1109/LICS.2017.8005100

LICS17MDPAccepted author manuscript, 379 KBLicence: Other

Energy Efficient Control
Mayr, R.
EPSRC
1/09/15 → 31/08/19
Project: Research

Richard Mayr
- Mayr36.teaching-admined.acuk
Person: Academic: Research Active

Cite this

@inproceedings{860e66fa3ef04c3380e4b8a8e8307930,

title = "Parity Objectives in Countable MDPs",

abstract = "We study countably infinite MDPs with parity objectives, and special cases with a bounded number of colors in the Mostowski hierarchy (including reachability, safety, Buchi and co-Buchi).In finite MDPs there always exist optimal memoryless deterministic (MD) strategies for parity objectives, but this does not generally hold for countably infinite MDPs. In particular, optimal strategies need not exist. For countable infinite MDPs, we provide a complete picture of the memory requirements of optimal (resp., ϵ-optimal) strategies for all objectives in the Mostowski hierarchy. In particular, there is a strong dichotomy between two different types of objectives. For the first type, optimal strategies, if they exist, can be chosen MD, while for the second type optimal strategies require infinite memory. (I.e., for all objectives in the Mostowski hierarchy, if finite-memory randomized strategies suffice then also MD strategies suffice.) Similarly, some objectives admit ϵ-optimal MD-strategies, while for others ϵ-optimal strategies require infinite memory. Such a dichotomy also holds for the subclass of countably infinite MDPs that are finitely branching, though more objectives admit MD-strategies here. ",

keywords = "countable MDPs, parity objectives, strategies, memory requirement",

author = "Richard Mayr and Stefan Kiefer and Mahsa Shirmohammadi and Dominik Wojtczak",

year = "2017",

month = aug,

day = "18",

doi = "10.1109/LICS.2017.8005100",

language = "English",

booktitle = "2017 32nd Annual ACM/IEEE Symposium on Logic in Computer Science (LICS)",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

address = "United States",

note = "2017 32nd Annual ACM/IEEE Symposium on Logic in Computer Science, LICS 2017 ; Conference date: 20-06-2017 Through 23-06-2017",

url = "http://lics.siglog.org/lics17/",

}

TY - GEN

T1 - Parity Objectives in Countable MDPs

AU - Mayr, Richard

AU - Kiefer, Stefan

AU - Shirmohammadi, Mahsa

AU - Wojtczak, Dominik

PY - 2017/8/18

Y1 - 2017/8/18

N2 - We study countably infinite MDPs with parity objectives, and special cases with a bounded number of colors in the Mostowski hierarchy (including reachability, safety, Buchi and co-Buchi).In finite MDPs there always exist optimal memoryless deterministic (MD) strategies for parity objectives, but this does not generally hold for countably infinite MDPs. In particular, optimal strategies need not exist. For countable infinite MDPs, we provide a complete picture of the memory requirements of optimal (resp., ϵ-optimal) strategies for all objectives in the Mostowski hierarchy. In particular, there is a strong dichotomy between two different types of objectives. For the first type, optimal strategies, if they exist, can be chosen MD, while for the second type optimal strategies require infinite memory. (I.e., for all objectives in the Mostowski hierarchy, if finite-memory randomized strategies suffice then also MD strategies suffice.) Similarly, some objectives admit ϵ-optimal MD-strategies, while for others ϵ-optimal strategies require infinite memory. Such a dichotomy also holds for the subclass of countably infinite MDPs that are finitely branching, though more objectives admit MD-strategies here.

AB - We study countably infinite MDPs with parity objectives, and special cases with a bounded number of colors in the Mostowski hierarchy (including reachability, safety, Buchi and co-Buchi).In finite MDPs there always exist optimal memoryless deterministic (MD) strategies for parity objectives, but this does not generally hold for countably infinite MDPs. In particular, optimal strategies need not exist. For countable infinite MDPs, we provide a complete picture of the memory requirements of optimal (resp., ϵ-optimal) strategies for all objectives in the Mostowski hierarchy. In particular, there is a strong dichotomy between two different types of objectives. For the first type, optimal strategies, if they exist, can be chosen MD, while for the second type optimal strategies require infinite memory. (I.e., for all objectives in the Mostowski hierarchy, if finite-memory randomized strategies suffice then also MD strategies suffice.) Similarly, some objectives admit ϵ-optimal MD-strategies, while for others ϵ-optimal strategies require infinite memory. Such a dichotomy also holds for the subclass of countably infinite MDPs that are finitely branching, though more objectives admit MD-strategies here.

KW - countable MDPs, parity objectives, strategies, memory requirement

U2 - 10.1109/LICS.2017.8005100

DO - 10.1109/LICS.2017.8005100

M3 - Conference contribution

BT - 2017 32nd Annual ACM/IEEE Symposium on Logic in Computer Science (LICS)

PB - Institute of Electrical and Electronics Engineers (IEEE)

T2 - 2017 32nd Annual ACM/IEEE Symposium on Logic in Computer Science

Y2 - 20 June 2017 through 23 June 2017

ER -

Parity Objectives in Countable MDPs

Abstract

Conference

Keywords

Access to Document

Fingerprint

Projects

Energy Efficient Control

Profiles

Richard Mayr

Cite this