A cultural algorithm for pomdps from stochastic inventory control

S.D. Prestwich, R. Rossi, S.A. Tarim, B. Hnich

Research output: Chapter in Book/Report/Conference proceedingChapter (peer-reviewed)peer-review


Reinforcement Learning algorithms such as SARSA with an eligibility trace, and Evolutionary Computation methods such as genetic algorithms, are competing approaches to solving Partially Observable Markov Decision Processes (POMDPs) which occur in many fields of Artificial Intelligence. A powerful form of evolutionary algorithm that has not previously been applied to POMDPs is the cultural algorithm, in which evolving agents share knowledge in a belief space that is used to guide their evolution. We describe a cultural algorithm for POMDPs that hybridises SARSA with a noisy genetic algorithm, and inherits the latter's convergence properties. Its belief space is a common set of state-action values that are updated during genetic exploration, and conversely used to modify chromosomes. We use it to solve problems from stochastic inventory control by finding memoryless policies for nondeterministic POMDPs. Neither SARSA nor the genetic algorithm dominates the other on these problems, but the cultural algorithm outperforms the genetic algorithm, and on highly non-Markovian instances also outperforms SARSA.
Original languageEnglish
Title of host publicationHybrid Metaheuristics
Subtitle of host publication5th International Workshop, HM 2008, Málaga, Spain, October 8-9, 2008. Proceedings
EditorsMaría J. Blesa, Christian Blum, Carlos Cotta, Antonio J. Fernández, José E. Gallardo, Andrea Roli, Michael Sampels
PublisherSpringer-Verlag GmbH
Number of pages13
Volume5296 LNCS
Publication statusPublished - 5 Oct 2008

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
PublisherSpringer Berlin / Heidelberg
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Dive into the research topics of 'A cultural algorithm for pomdps from stochastic inventory control'. Together they form a unique fingerprint.

Cite this