A Reconfigurable Fabric for Accelerating Large-scale Datacenter Services

Andrew Putnam; Adrian M. Caulfield; Eric S. Chung; Derek Chiou; Kypros Constantinides; John Demme; Hadi Esmaeilzadeh; Jeremy Fowers; Gopi Prashanth Gopal; Jan Gray; Michael Haselman; Scott Hauck; Stephen Heil; Amir Hormati; Joo-Young Kim; Sitaram Lanka; James Larus; Eric Peterson; Simon Pope; Aaron Smith; Jason Thong; Phillip Yi Xiao; Doug Burger

doi:10.1145/2678373.2665678

A Reconfigurable Fabric for Accelerating Large-scale Datacenter Services

Andrew Putnam, Adrian M. Caulfield, Eric S. Chung, Derek Chiou, Kypros Constantinides, John Demme, Hadi Esmaeilzadeh, Jeremy Fowers, Gopi Prashanth Gopal, Jan Gray, Michael Haselman, Scott Hauck, Stephen Heil, Amir Hormati, Joo-Young Kim, Sitaram Lanka, James Larus, Eric Peterson, Simon Pope, Aaron SmithJason Thong, Phillip Yi Xiao, Doug Burger

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Datacenter workloads demand high computational capabilities, flexibility, power efficiency, and low cost. It is challenging to improve all of these factors simultaneously. To advance datacenter capabilities beyond what commodity server designs can provide, we have designed and built a composable, reconfigurablefabric to accelerate portions of large-scale software services. Each instantiation of the fabric consists of a 6x8 2-D torus of high-end Stratix V FPGAs embedded into a half-rack of 48 machines. One FPGA is placed into each server, accessible through PCIe, and wired directly to other FPGAs with pairs of 10 Gb SAS cables

In this paper, we describe a medium-scale deployment of this fabric on a bed of 1,632 servers, and measure its efficacy in accelerating the Bing web search engine. We describe the requirements and architecture of the system, detail the critical engineering challenges and solutions needed to make the system robust in the presence of failures, and measure the performance, power, and resilience of the system when ranking candidate documents. Under high load, the largescale reconfigurable fabric improves the ranking throughput of each server by a factor of 95% for a fixed latency distribution--- or, while maintaining equivalent throughput, reduces the tail latency by 29%

Original language	English
Title of host publication	Proceeding of the 41st Annual International Symposium on Computer Architecuture
Place of Publication	Piscataway, NJ, USA
Publisher	Institute of Electrical and Electronics Engineers
Pages	13-24
Number of pages	12
ISBN (Print)	978-1-4799-4394-4
DOIs	https://doi.org/10.1145/2678373.2665678
Publication status	Published - Jun 2014

Publication series

Name	ISCA '14
Publisher	IEEE Press

Access to Document

10.1145/2678373.2665678

http://dl.acm.org/citation.cfm?id=2665671.2665678

Cite this

Putnam, A., Caulfield, A. M., Chung, E. S., Chiou, D., Constantinides, K., Demme, J., Esmaeilzadeh, H., Fowers, J., Gopal, G. P., Gray, J., Haselman, M., Hauck, S., Heil, S., Hormati, A., Kim, J.-Y., Lanka, S., Larus, J., Peterson, E., Pope, S., ... Burger, D. (2014). A Reconfigurable Fabric for Accelerating Large-scale Datacenter Services. In Proceeding of the 41st Annual International Symposium on Computer Architecuture (pp. 13-24). (ISCA '14). Institute of Electrical and Electronics Engineers. https://doi.org/10.1145/2678373.2665678

@inproceedings{ff72dc682f8d42b2bb7b3fa7b704d409,

title = "A Reconfigurable Fabric for Accelerating Large-scale Datacenter Services",

abstract = "Datacenter workloads demand high computational capabilities, flexibility, power efficiency, and low cost. It is challenging to improve all of these factors simultaneously. To advance datacenter capabilities beyond what commodity server designs can provide, we have designed and built a composable, reconfigurablefabric to accelerate portions of large-scale software services. Each instantiation of the fabric consists of a 6x8 2-D torus of high-end Stratix V FPGAs embedded into a half-rack of 48 machines. One FPGA is placed into each server, accessible through PCIe, and wired directly to other FPGAs with pairs of 10 Gb SAS cablesIn this paper, we describe a medium-scale deployment of this fabric on a bed of 1,632 servers, and measure its efficacy in accelerating the Bing web search engine. We describe the requirements and architecture of the system, detail the critical engineering challenges and solutions needed to make the system robust in the presence of failures, and measure the performance, power, and resilience of the system when ranking candidate documents. Under high load, the largescale reconfigurable fabric improves the ranking throughput of each server by a factor of 95% for a fixed latency distribution--- or, while maintaining equivalent throughput, reduces the tail latency by 29%",

author = "Andrew Putnam and Caulfield, {Adrian M.} and Chung, {Eric S.} and Derek Chiou and Kypros Constantinides and John Demme and Hadi Esmaeilzadeh and Jeremy Fowers and Gopal, {Gopi Prashanth} and Jan Gray and Michael Haselman and Scott Hauck and Stephen Heil and Amir Hormati and Joo-Young Kim and Sitaram Lanka and James Larus and Eric Peterson and Simon Pope and Aaron Smith and Jason Thong and Xiao, {Phillip Yi} and Doug Burger",

year = "2014",

month = jun,

doi = "10.1145/2678373.2665678",

language = "English",

isbn = "978-1-4799-4394-4",

series = "ISCA '14",

publisher = "Institute of Electrical and Electronics Engineers",

pages = "13--24",

booktitle = "Proceeding of the 41st Annual International Symposium on Computer Architecuture",

address = "United States",

}

Putnam, A, Caulfield, AM, Chung, ES, Chiou, D, Constantinides, K, Demme, J, Esmaeilzadeh, H, Fowers, J, Gopal, GP, Gray, J, Haselman, M, Hauck, S, Heil, S, Hormati, A, Kim, J-Y, Lanka, S, Larus, J, Peterson, E, Pope, S, Smith, A, Thong, J, Xiao, PY & Burger, D 2014, A Reconfigurable Fabric for Accelerating Large-scale Datacenter Services. in Proceeding of the 41st Annual International Symposium on Computer Architecuture. ISCA '14, Institute of Electrical and Electronics Engineers, Piscataway, NJ, USA, pp. 13-24. https://doi.org/10.1145/2678373.2665678

A Reconfigurable Fabric for Accelerating Large-scale Datacenter Services. / Putnam, Andrew; Caulfield, Adrian M.; Chung, Eric S. et al.
Proceeding of the 41st Annual International Symposium on Computer Architecuture. Piscataway, NJ, USA: Institute of Electrical and Electronics Engineers, 2014. p. 13-24 (ISCA '14).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - A Reconfigurable Fabric for Accelerating Large-scale Datacenter Services

AU - Putnam, Andrew

AU - Caulfield, Adrian M.

AU - Chung, Eric S.

AU - Chiou, Derek

AU - Constantinides, Kypros

AU - Demme, John

AU - Esmaeilzadeh, Hadi

AU - Fowers, Jeremy

AU - Gopal, Gopi Prashanth

AU - Gray, Jan

AU - Haselman, Michael

AU - Hauck, Scott

AU - Heil, Stephen

AU - Hormati, Amir

AU - Kim, Joo-Young

AU - Lanka, Sitaram

AU - Larus, James

AU - Peterson, Eric

AU - Pope, Simon

AU - Smith, Aaron

AU - Thong, Jason

AU - Xiao, Phillip Yi

AU - Burger, Doug

PY - 2014/6

Y1 - 2014/6

N2 - Datacenter workloads demand high computational capabilities, flexibility, power efficiency, and low cost. It is challenging to improve all of these factors simultaneously. To advance datacenter capabilities beyond what commodity server designs can provide, we have designed and built a composable, reconfigurablefabric to accelerate portions of large-scale software services. Each instantiation of the fabric consists of a 6x8 2-D torus of high-end Stratix V FPGAs embedded into a half-rack of 48 machines. One FPGA is placed into each server, accessible through PCIe, and wired directly to other FPGAs with pairs of 10 Gb SAS cablesIn this paper, we describe a medium-scale deployment of this fabric on a bed of 1,632 servers, and measure its efficacy in accelerating the Bing web search engine. We describe the requirements and architecture of the system, detail the critical engineering challenges and solutions needed to make the system robust in the presence of failures, and measure the performance, power, and resilience of the system when ranking candidate documents. Under high load, the largescale reconfigurable fabric improves the ranking throughput of each server by a factor of 95% for a fixed latency distribution--- or, while maintaining equivalent throughput, reduces the tail latency by 29%

AB - Datacenter workloads demand high computational capabilities, flexibility, power efficiency, and low cost. It is challenging to improve all of these factors simultaneously. To advance datacenter capabilities beyond what commodity server designs can provide, we have designed and built a composable, reconfigurablefabric to accelerate portions of large-scale software services. Each instantiation of the fabric consists of a 6x8 2-D torus of high-end Stratix V FPGAs embedded into a half-rack of 48 machines. One FPGA is placed into each server, accessible through PCIe, and wired directly to other FPGAs with pairs of 10 Gb SAS cablesIn this paper, we describe a medium-scale deployment of this fabric on a bed of 1,632 servers, and measure its efficacy in accelerating the Bing web search engine. We describe the requirements and architecture of the system, detail the critical engineering challenges and solutions needed to make the system robust in the presence of failures, and measure the performance, power, and resilience of the system when ranking candidate documents. Under high load, the largescale reconfigurable fabric improves the ranking throughput of each server by a factor of 95% for a fixed latency distribution--- or, while maintaining equivalent throughput, reduces the tail latency by 29%

U2 - 10.1145/2678373.2665678

DO - 10.1145/2678373.2665678

M3 - Conference contribution

SN - 978-1-4799-4394-4

T3 - ISCA '14

SP - 13

EP - 24

BT - Proceeding of the 41st Annual International Symposium on Computer Architecuture

PB - Institute of Electrical and Electronics Engineers

CY - Piscataway, NJ, USA

ER -

A Reconfigurable Fabric for Accelerating Large-scale Datacenter Services

Abstract

Publication series

Access to Document

Fingerprint

Cite this