NVRAM as an enabler to new horizons in graph processing

Ludovic Anthony Richard Capelli*, Nicholas Brown, Jonathan Mark Bull

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

From the world wide web, to genomics, to traffic analysis, graphs are central to many scientific, engineering, and societal endeavours. Therefore an important question is what hardware technologies are most appropriate to invest in and use for processing graphs, whose sizes now frequently reach terabytes. Non-Volatile Random Access Memory (NVRAM) technology is an interesting candidate enabling organisations to extend the memory in their systems typically by an order of magnitude compared to Dynamic Read Access Memory (DRAM) alone. From a software perspective, it permits to store a much larger dataset within a single memory space and avoid the significant communication cost incurred when going off node. However, to obtain optimal performance one must consider carefully how best to integrate this technology with their code to cope with NVRAM esoteric properties such as asymmetric read/write performance or explicit coding for deeper memory hierarchies for instance.

In this paper, we investigate the use of NVRAM in the context of shared memory graph processing via vertex-centric. We find that NVRAM enables the processing of exceptionally large graphs on a single node with good performance, price and power consumption. We also explore the techniques required to most appropriately exploit NVRAM for graph processing and, for the first time, demonstrate the ability to process a graph of 750 billion edges whilst staying within the memory of a single node. Whilst the vertex-centric graph processing methodology is our main focus, not least due to its popularity since introduced by Google over a decade ago, the lessons learnt in this paper apply more widely to graph processing in general.
Original languageEnglish
Article number385
Number of pages13
JournalSN Computer Science
Volume3
Issue number5
DOIs
Publication statusPublished - 20 Jul 2022

Keywords

  • NVRAM
  • DCPMM
  • vertex-centric

Fingerprint

Dive into the research topics of 'NVRAM as an enabler to new horizons in graph processing'. Together they form a unique fingerprint.

Cite this