Permissive Finite-State Controllers of POMDPs using Parameter Synthesis

Abstract

We study finite-state controllers (FSCs) for partially observable Markov decision processes (POMDPs). The key insight is that computing (randomized) FSCs on POMDPs is equivalent to synthesis for parametric Markov chains (pMCs). This correspondence enables using parameter synthesis techniques to compute FSCs for POMDPs in a black-box fashion. We investigate how typical restrictions on parameter values affect the quality of the obtained FSCs. Permissive strategies for POMDPs are obtained as regions of parameter values, a natural output of parameter synthesis techniques. Experimental evaluation on several POMDP benchmarks shows promising results.