Self-managed collections: Off-heap memory management for scalable query-dominated collections

Fabian Nagel, Gavin M. Bierman, Aleksandar Dragojevic, Stratis Viglas

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Explosive growth in DRAM capacities and the emergence of language-integrated query enable a new class of managed applications that perform complex query processing on huge volumes of data stored as collections of objects in the memory space of the application. While more flexible in terms of schema design and application development, this approach typically experiences sub-par query execution performance when compared to specialized systems like DBMS.

To address this issue, we propose self-managed collections, which utilize off-heap memory management and dynamic query compilation to improve the performance of querying managed data through language-integrated query. We evaluate self-managed collections using both microbenchmarks and enumeration-heavy queries from the TPC-H business intelligence benchmark. Our results show that self-managed collections outperform ordinary managed collections in both query processing and memory management by up to an order of magnitude and even outperform an optimized in memory columnar database system for the vast majority of queries.
Original languageEnglish
Title of host publicationProc. 20th International Conference on Extending Database Technology (EDBT)
PublisherOpenProceedings
Pages61-71
Number of pages11
ISBN (Print)978-3-89318-073-8
DOIs
Publication statusE-pub ahead of print - 24 Mar 2017
Event20th International Conference on Extending Database Technology - Venice, Italy
Duration: 21 Mar 201724 Mar 2017
http://edbticdt2017.unive.it/

Publication series

NameOpen Proceedings
PublisherOpenProceedings.org
ISSN (Print)2367-2005

Conference

Conference20th International Conference on Extending Database Technology
Abbreviated titleEDBT 2017
Country/TerritoryItaly
CityVenice
Period21/03/1724/03/17
Internet address

Fingerprint

Dive into the research topics of 'Self-managed collections: Off-heap memory management for scalable query-dominated collections'. Together they form a unique fingerprint.

Cite this