Fault Localization for Buggy Deep Learning Framework Conversions in Image Recognition

Nick Louloudakis, Perry Gibson, Jose Cano, Ajitha Rajan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

When deploying Deep Neural Networks (DNNs), developers often convert models from one deep learning framework to another (e.g., TensorFlow to PyTorch). However, this process is error-prone and can impact target model accuracy. To identify the extent of such impact, we perform and briefly present a differential analysis against three DNNs widely used for image recognition (MobileNetV2, ResNet101, and InceptionV3) converted across four well-known deep learning frameworks (PyTorch, Keras, TensorFlow (TF), and TFLite), which revealed numerous model crashes and output label discrepancies of up to 72%. To mitigate such errors, we present a novel approach towards fault localization and repair of buggy deep learning framework conversions, focusing on pre-trained image recognition models. Our technique consists of four stages of analysis: 1) conversion tools, 2) model parameters, 3) model hyperparameters, and 4) graph representation. In addition, we propose various strategies towards fault repair of the faults detected. We implement our technique on top of the Apache TVM deep learning compiler, and we test it by conducting a preliminary fault localization analysis for the conversion of InceptionV3 from TF to TFLite. Our approach detected a fault in a common DNN converter tool, which introduced precision errors in weights, reducing model accuracy. After our fault localization, we repaired the issue, reducing our conversion error to zero.
Original languageEnglish
Title of host publication38th IEEE/ACM International Conference on Automated Software Engineering
PublisherIEEE
Pages1795-1799
Number of pages5
ISBN (Electronic)979-8-3503-2996-4
ISBN (Print)979-8-3503-2997-1
DOIs
Publication statusPublished - 8 Nov 2023
Event38th IEEE/ACM International Conference on Automated Software Engineering - Kirchberg, Luxembourg
Duration: 11 Sept 202315 Sept 2023
Conference number: 38
https://conf.researchr.org/home/ase-2023

Publication series

NameIEEE/ACM International Conference on Automated Software Engineering (ASE)
ISSN (Print)1938-4300
ISSN (Electronic)2643-1572

Conference

Conference38th IEEE/ACM International Conference on Automated Software Engineering
Abbreviated titleASE 2023
Country/TerritoryLuxembourg
CityKirchberg
Period11/09/2315/09/23
Internet address

Fingerprint

Dive into the research topics of 'Fault Localization for Buggy Deep Learning Framework Conversions in Image Recognition'. Together they form a unique fingerprint.

Cite this