Fex: A Software Systems Evaluator

Oleksii Oleksenko, Dmitrii Kuvaiskii, Pramod Bhatotia, Christof Fetzer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Software systems research relies on experimental evaluation to assess the effectiveness of newly developed solutions. However, the existing evaluation frameworks are rigid (do not allow creation of new experiments), often
simplistic (may not reveal issues that appear in real-world applications), and can be inconsistent (do not guarantee reproducibility of experiments across platforms).
This paper presents FEX, a software systems evaluation framework that addresses these limitations. FEX is extensible (can be easily extended with custom experiment types), practical (supports composition of different benchmark suites and real-world applications), and reproducible (it is built
on container technology to guarantee the same software stack across platforms). We show that FEX achieves these design goals with minimal end-user effort—for instance, adding Nginx web-server to evaluation requires only 160 LoC. Going forward, we discuss the architecture of the framework, explain its interface, show common usage scenarios, and evaluate the efforts for writing various custom extensions.
Original languageEnglish
Title of host publicationThe 47th IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2017)
Place of PublicationDenver, CO, USA
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages543-550
Number of pages8
ISBN (Electronic)978-1-5386-0542-4
ISBN (Print)978-1-5386-0543-1
DOIs
Publication statusPublished - 31 Aug 2017
Event47th IEEE/IFIP International Conference on Dependable Systems and Networks - Denver, United States
Duration: 26 Jun 201729 Jun 2017
http://sn.committees.comsoc.org/call-for-papers/dsn-2017/

Publication series

Name
PublisherIEEE
ISSN (Electronic)2158-3927

Conference

Conference47th IEEE/IFIP International Conference on Dependable Systems and Networks
Abbreviated titleDSN 2017
CountryUnited States
CityDenver
Period26/06/1729/06/17
Internet address

Fingerprint

Dive into the research topics of 'Fex: A Software Systems Evaluator'. Together they form a unique fingerprint.

Cite this