Projects per year
We present dispel4py, a novel data intensive and high performance computing middleware provided as a standard Python library for describing stream-based workflows. It allows its users to develop their scientific applications locally and then run them on a wide range of HPC-infrastructures without any changes to the code. Moreover, it provides automated and efficient parallel mappings to MPI, multiprocessing, Storm and Spark frameworks, commonly used in big data applications. It builds on the wide availability of Python in many environments and only requires familiarity with basic Python syntax. We will show the dispel4py advantages by walking through an example. We will conclude demonstrating how dispel4py can be employed as an easy-to-use tool for designing scientific applications using real-world scenarios.
|Title of host publication||Proceedings of the 5th Workshop on Python for High-Performance and Scientific Computing|
|Place of Publication||New York, NY, USA|
|Number of pages||10|
|Publication status||Published - 2015|
FingerprintDive into the research topics of 'Dispel4Py: A Python Framework for Data-intensive eScience'. Together they form a unique fingerprint.
- 2 Finished
14/11/13 → 14/10/15
Atkinson, M. & Parsons, M.
1/10/11 → 30/09/15