Dialogue-based Generation of Self-Driving Simulation Scenarios using Large Language Models

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Simulation is an invaluable tool for developing and evaluating controllers for self-driving cars. Current simulation frameworks are driven by highly-specialist domain specific languages, and so a natural language interface would greatly enhance usability. But there is often a gap, consisting of tacit assumptions the user is making, between a concise English utterance and the executable code that captures the user’s intent. In this paper we describe a system that addresses this issue by supporting an extended multimodal interaction: the user can follow up prior instructions with refinements or revisions, in reaction to the simulations that have been generated from their utterances so far. We use Large Language Models (LLMs) to map the user’s English utterances in this interaction into domain-specific code, and so we explore the extent to which LLMs capture the context sensitivity that’s necessary for computing the speaker’s intended message in discourse.
Original languageEnglish
Title of host publicationProceedings of the Third International Combined Workshop on Spatial Language Understanding and Grounded Communication for Robotics (SPLU-RoboNLP)
PublisherAssociation for Computational Linguistics
Pages1-12
Number of pages12
DOIs
Publication statusPublished - 6 Dec 2023
EventThird International Combined Workshop on Spatial Language Understanding and Grounded Communication for Robotics - , Singapore
Duration: 6 Dec 20237 Dec 2023
Conference number: 3
https://splu-robonlp-2023.github.io/

Workshop

WorkshopThird International Combined Workshop on Spatial Language Understanding and Grounded Communication for Robotics
Abbreviated titleSpLU-RoboNLP 2023
Country/TerritorySingapore
Period6/12/237/12/23
Internet address

Fingerprint

Dive into the research topics of 'Dialogue-based Generation of Self-Driving Simulation Scenarios using Large Language Models'. Together they form a unique fingerprint.

Cite this