Metric Space Magnitude and Generalisation in Neural Networks

Rayna Andreeva, Katharina Limbeck, Bastian Rieck, Rik Sarkar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Deep learning models have seen significant successes in numerous applications, but their inner workings remain elusive. The purpose of this work is to quantify the learning process of deep neural networks through the lens of a novel topological invariant called magnitude. Magnitude is an isometry invariant; its properties are an active area of research as it encodes many known invariants of a metric space. We use magnitude to study the internal representations of neural networks and propose a new method for determining their generalisation capabilities. Moreover, we theoretically connect magnitude dimension and the generalisation error, and demonstrate experimentally that the proposed framework can be a good indicator of the latter.
Original languageEnglish
Title of host publicationProceedings of 2nd Annual Workshop on Topology, Algebra, and Geometry in Machine Learning (TAG-ML)
PublisherJournal of Machine Learning Research: Workshop and Conference Proceedings
Pages242-253
Number of pages11
Volume221
EditionTAG-ML, ICML
Publication statusPublished - 28 Jul 2023
EventThe Fortieth International Conference on Machine Learning - Honolulu, United States
Duration: 23 Jul 202329 Jul 2023
Conference number: 40
https://icml.cc/

Publication series

NameProceedings of Machine Learning Research
PublisherMIT-Press
Volume221
ISSN (Electronic)2640-3498

Conference

ConferenceThe Fortieth International Conference on Machine Learning
Abbreviated titleICML 2023
Country/TerritoryUnited States
CityHonolulu
Period23/07/2329/07/23
Internet address

Fingerprint

Dive into the research topics of 'Metric Space Magnitude and Generalisation in Neural Networks'. Together they form a unique fingerprint.

Cite this