Skip to main navigation Skip to search Skip to main content

LexTime: A benchmark for temporal ordering of legal events

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Temporal reasoning in legal texts is important for applications like case law analysis and compliance monitoring. However, existing datasets lack expert language evaluation, leaving a gap in understanding how LLMs manage event ordering in legal contexts. We introduce LEXTIME, the first dataset designed to evaluate LLMs’ event ordering capabilities in legal language, consisting of 512 instances from U.S. Federal Complaints with annotated event pairs and their temporal relations. Our findings show that (1) LLMs are more accurate on legal event ordering than on narrative (up to +10.5%); (2) longer input contexts and implicit events boost accuracy, reaching 80.8% for implicit-explicit event pairs; (3) legal linguistic complexities and nested clauses remain a challenge. We investigate how context length, explicit vs implicit event pairs, and legal language features affect model performance, demonstrating the need for specific modeling strategies to enhance temporal event reasoning. The code and LEXTIME dataset are available at: https://github.com/clairebarale/LexTime.
Original languageEnglish
Title of host publicationFindings of the 2025 Conference on Empirical Methods in Natural Language Processing
PublisherAssociation for Computational Linguistics (ACL)
Pages1-16
Number of pages16
Publication statusPublished - Nov 2025
Event30th Conference on Empirical Methods in Natural Language Processing - Suzhou International Expo Centre, Suzhou, China
Duration: 4 Nov 2025 → 9 Nov 2025
Conference number: 30
https://2025.emnlp.org/

Conference

Conference30th Conference on Empirical Methods in Natural Language Processing
Abbreviated titleEMNLP 2025
Country/TerritoryChina
CitySuzhou
Period4/11/25 → 9/11/25
Internet address

Fingerprint

Dive into the research topics of 'LexTime: A benchmark for temporal ordering of legal events'. Together they form a unique fingerprint.

Cite this