Optimizing Grouped Convolutions on Edge Devices

Perry Gibson; Jose Cano; Jack Turner; Elliot J Crowley; Michael F P O'Boyle; Amos J Storkey

doi:10.1109/ASAP49362.2020.00039

Optimizing Grouped Convolutions on Edge Devices

Perry Gibson, Jose Cano, Jack Turner, Elliot J Crowley, Michael F P O'Boyle, Amos J Storkey

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

When deploying a deep neural network on con-strained hardware, it is possible to replace the network’s standard convolutions with grouped convolutions. This allows for substantial memory savings with minimal loss of accuracy. However, current implementations of grouped convolutions in modern deep learning frameworks are far from performing optimally in terms of speed. In this paper we propose Grouped Spatial Pack Convolutions (GSPC), a new implementation of grouped convolutions that outperforms existing solutions. We implement GSPC in TVM, which provides state-of-the-art performance on edge devices. We analyze a set of networks utilizing different types of grouped convolutions and evaluate their performance in terms of inference time on several edge devices. We observe that our new implementation scales well with the number of groups and provides the best inference times in all settings, improving the existing implementations of grouped convolutions in TVM, PyTorch and TensorFlow Lite by 3.4×, 8× and 4× on average respectively. Code is available at https://github.com/gecLAB/tvm-GSPC/

Original language	English
Title of host publication	2020 IEEE 31st International Conference on Application-specific Systems, Architectures and Processors (ASAP)
Publisher	Institute of Electrical and Electronics Engineers
Pages	189 - 196
Number of pages	8
ISBN (Electronic)	978-1-7281-7147-0
ISBN (Print)	978-1-7281-7279-8
DOIs	https://doi.org/10.1109/ASAP49362.2020.00039
Publication status	Published - 31 Jul 2020
Event	31st IEEE International Conference on Application-specific Systems, Architectures and Processors - The University of Manchester, Machester, United Kingdom Duration: 6 Jul 2020 → 8 Jul 2020 https://asap2020.cs.manchester.ac.uk/index.php

Publication series

Name
Publisher	IEEE
ISSN (Print)	2160-0511
ISSN (Electronic)	2160-052X

Conference

Conference	31st IEEE International Conference on Application-specific Systems, Architectures and Processors
Abbreviated title	ASAP 2020
Country/Territory	United Kingdom
City	Machester
Period	6/07/20 → 8/07/20
Internet address	https://asap2020.cs.manchester.ac.uk/index.php

Access to Document

10.1109/ASAP49362.2020.00039Licence: All Rights Reserved

Optimizing Grouped_GIBSON_DOA20052020_AFVAccepted author manuscript, 1.33 MBLicence: Creative Commons: Attribution-NonCommercial-ShareAlike (CC BY-NC-SA)

http://eprints.gla.ac.uk/221586/Licence: Creative Commons: Attribution-NonCommercial-ShareAlike (CC BY-NC-SA)
https://ieeexplore.ieee.org/document/9153227Licence: All Rights Reserved

Bonseyes - Platform for Open Development of Systems of Artificial Intelligence
Storkey, A. & O'Boyle, M.
EU government bodies
1/12/16 → 31/01/20
Project: Research

Cite this

@inproceedings{7d25373af45042bcbd4cfbc152dab67a,

title = "Optimizing Grouped Convolutions on Edge Devices",

abstract = "When deploying a deep neural network on con-strained hardware, it is possible to replace the network{\textquoteright}s standard convolutions with grouped convolutions. This allows for substantial memory savings with minimal loss of accuracy. However, current implementations of grouped convolutions in modern deep learning frameworks are far from performing optimally in terms of speed. In this paper we propose Grouped Spatial Pack Convolutions (GSPC), a new implementation of grouped convolutions that outperforms existing solutions. We implement GSPC in TVM, which provides state-of-the-art performance on edge devices. We analyze a set of networks utilizing different types of grouped convolutions and evaluate their performance in terms of inference time on several edge devices. We observe that our new implementation scales well with the number of groups and provides the best inference times in all settings, improving the existing implementations of grouped convolutions in TVM, PyTorch and TensorFlow Lite by 3.4×, 8× and 4× on average respectively. Code is available at https://github.com/gecLAB/tvm-GSPC/",

author = "Perry Gibson and Jose Cano and Jack Turner and Crowley, {Elliot J} and O'Boyle, {Michael F P} and Storkey, {Amos J}",

year = "2020",

month = jul,

day = "31",

doi = "10.1109/ASAP49362.2020.00039",

language = "English",

isbn = "978-1-7281-7279-8",

publisher = "Institute of Electrical and Electronics Engineers",

pages = "189 -- 196",

booktitle = "2020 IEEE 31st International Conference on Application-specific Systems, Architectures and Processors (ASAP)",

address = "United States",

note = "31st IEEE International Conference on Application-specific Systems, Architectures and Processors, ASAP 2020 ; Conference date: 06-07-2020 Through 08-07-2020",

url = "https://asap2020.cs.manchester.ac.uk/index.php",

}

Gibson, P, Cano, J, Turner, J, Crowley, EJ , O'Boyle, MFP & Storkey, AJ 2020, Optimizing Grouped Convolutions on Edge Devices. in 2020 IEEE 31st International Conference on Application-specific Systems, Architectures and Processors (ASAP). Institute of Electrical and Electronics Engineers, pp. 189 - 196, 31st IEEE International Conference on Application-specific Systems, Architectures and Processors, Machester, United Kingdom, 6/07/20. https://doi.org/10.1109/ASAP49362.2020.00039

TY - GEN

T1 - Optimizing Grouped Convolutions on Edge Devices

AU - Gibson, Perry

AU - Cano, Jose

AU - Turner, Jack

AU - Crowley, Elliot J

AU - O'Boyle, Michael F P

AU - Storkey, Amos J

PY - 2020/7/31

Y1 - 2020/7/31

N2 - When deploying a deep neural network on con-strained hardware, it is possible to replace the network’s standard convolutions with grouped convolutions. This allows for substantial memory savings with minimal loss of accuracy. However, current implementations of grouped convolutions in modern deep learning frameworks are far from performing optimally in terms of speed. In this paper we propose Grouped Spatial Pack Convolutions (GSPC), a new implementation of grouped convolutions that outperforms existing solutions. We implement GSPC in TVM, which provides state-of-the-art performance on edge devices. We analyze a set of networks utilizing different types of grouped convolutions and evaluate their performance in terms of inference time on several edge devices. We observe that our new implementation scales well with the number of groups and provides the best inference times in all settings, improving the existing implementations of grouped convolutions in TVM, PyTorch and TensorFlow Lite by 3.4×, 8× and 4× on average respectively. Code is available at https://github.com/gecLAB/tvm-GSPC/

AB - When deploying a deep neural network on con-strained hardware, it is possible to replace the network’s standard convolutions with grouped convolutions. This allows for substantial memory savings with minimal loss of accuracy. However, current implementations of grouped convolutions in modern deep learning frameworks are far from performing optimally in terms of speed. In this paper we propose Grouped Spatial Pack Convolutions (GSPC), a new implementation of grouped convolutions that outperforms existing solutions. We implement GSPC in TVM, which provides state-of-the-art performance on edge devices. We analyze a set of networks utilizing different types of grouped convolutions and evaluate their performance in terms of inference time on several edge devices. We observe that our new implementation scales well with the number of groups and provides the best inference times in all settings, improving the existing implementations of grouped convolutions in TVM, PyTorch and TensorFlow Lite by 3.4×, 8× and 4× on average respectively. Code is available at https://github.com/gecLAB/tvm-GSPC/

U2 - 10.1109/ASAP49362.2020.00039

DO - 10.1109/ASAP49362.2020.00039

M3 - Conference contribution

SN - 978-1-7281-7279-8

SP - 189

EP - 196

BT - 2020 IEEE 31st International Conference on Application-specific Systems, Architectures and Processors (ASAP)

PB - Institute of Electrical and Electronics Engineers

T2 - 31st IEEE International Conference on Application-specific Systems, Architectures and Processors

Y2 - 6 July 2020 through 8 July 2020

ER -

Optimizing Grouped Convolutions on Edge Devices

Abstract

Publication series

Conference

Access to Document

Fingerprint

Projects

Bonseyes - Platform for Open Development of Systems of Artificial Intelligence

Cite this