Reinforcement learning of dialogue strategies with hierarchical abstract machines

Heriberto Cuayáhuitl, Steve Renals, Oliver Lemon, Hiroshi Shimodaira

Research output: Chapter in Book/Report/Conference proceedingConference contribution


In this paper we propose partially specified dialogue strategies for dialogue strategy optimization, where part of the strategy is specified deterministically and the rest optimized with reinforcement learning (RL). To do this we apply RL with hierarchical abstract machines (HAMs). We also propose to build simulated users using HAMs, incorporating a combination of hierarchical deterministic and probabilistic behaviour. We performed experiments using a single-goal flight booking dialogue system, and compare two dialogue strategies (deterministic and optimized) using three types of simulated user (novice, experienced and expert). Our results show that HAMs are promising for both dialogue optimization and simulation, and provide evidence that indeed partially specified dialogue strategies can outperform deterministic ones (on average 4.7 fewer system turns) with faster learning than the traditional RL framework.
Original languageEnglish
Title of host publication2006 IEEE Spoken Language Technology Workshop
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Number of pages4
ISBN (Print)1-4244-0872-5
Publication statusPublished - 2006
EventIEEE ACL Spoken Language Technology Workshop (SLT 2006) - Aruba Marriott, Palm Beach, Aruba
Duration: 10 Dec 200613 Dec 2006


WorkshopIEEE ACL Spoken Language Technology Workshop (SLT 2006)
CityPalm Beach

Fingerprint Dive into the research topics of 'Reinforcement learning of dialogue strategies with hierarchical abstract machines'. Together they form a unique fingerprint.

Cite this