Scalable Querying of Nested Data

Jaclyn Smith, Michael Benedikt, Milos Nikolic, Amir Shaikhha

Research output: Contribution to journalArticlepeer-review


While large-scale distributed data processing platforms have become an attractive target for query processing, these systems are problematic for applications that deal with nested collections. Programmers are forced either to perform non-trivial translations of collection programs or to employ automated flattening procedures, both of which lead to performance problems. These challenges only worsen for nested collections with skewed cardinalities, where both handcrafted rewriting and automated flattening are unable to enforce load balancing across partitions.
In this work, we propose a framework that translates a program manipulating nested collections into a set of semantically equivalent shredded queries that can be efficiently evaluated. The framework employs a combination of query compilation techniques, an efficient data representation for nested collections, and automated skew-handling. We provide an extensive experimental evaluation, demonstrating significant improvements provided by the framework in diverse scenarios for nested collection programs.
Original languageEnglish
Pages (from-to)445-457
Number of pages13
JournalProceedings of the VLDB Endowment (PVLDB)
Issue number3
Publication statusPublished - 30 Nov 2020

Fingerprint Dive into the research topics of 'Scalable Querying of Nested Data'. Together they form a unique fingerprint.

Cite this