Ε(2)-Equivariant Vision Transformer

Renjun Xu; Kaifan Yang; Ke Liu; Fengxiang He

Ε(2)-Equivariant Vision Transformer

Renjun Xu, Kaifan Yang, Ke Liu^*, Fengxiang He

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Vision Transformer (ViT) has achieved remarkable performance in computer vision. However, positional encoding in ViT makes it substantially difficult to learn the intrinsic equivariance in data. Initial attempts have been made on designing equivariant ViT but are proved defective in some cases in this paper. To address this issue, we design a Group Equivariant Vision Transformer (GE-ViT) via a novel, effective positional encoding operator. We prove that GE-ViT meets all the theoretical requirements of an equivariant neural network. Comprehensive experiments are conducted on standard benchmark datasets, demonstrating that GE-ViT significantly outperforms non-equivariant self-attention networks. The code is available at https://github.com/ZJUCDSYangKaifan/GEVit.

Original language	English
Title of host publication	Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence
Editors	Robin J. Evans, Ilya Shpitser
Publisher	PMLR
Pages	2356-2366
Number of pages	11
Volume	216
Publication status	Published - 1 Jul 2023
Event	39th Conference on Uncertainty in Artificial Intelligence - Pittsburgh, United States Duration: 31 Jul 2023 → 4 Aug 2023 Conference number: 39 https://www.auai.org/uai2023/

Publication series

Name	Proceedings of Machine Learning Research
Publisher	PMLR
ISSN (Electronic)	2640-3498

Conference

Conference	39th Conference on Uncertainty in Artificial Intelligence
Abbreviated title	UAI 2023
Country/Territory	United States
City	Pittsburgh
Period	31/07/23 → 4/08/23
Internet address	https://www.auai.org/uai2023/

Access to Document

E(2)-Equivariant Vision Transformer_XU_DOA08052023_VOR_CC BYFinal published version, 776 KBLicence: Creative Commons: Attribution (CC-BY)

https://proceedings.mlr.press/v216/xu23b.htmlLicence: Creative Commons: Attribution (CC-BY)

Cite this

@inproceedings{53c321d033ba46baaf3db4a73e3cb49e,

title = "Ε(2)-Equivariant Vision Transformer",

abstract = "Vision Transformer (ViT) has achieved remarkable performance in computer vision. However, positional encoding in ViT makes it substantially difficult to learn the intrinsic equivariance in data. Initial attempts have been made on designing equivariant ViT but are proved defective in some cases in this paper. To address this issue, we design a Group Equivariant Vision Transformer (GE-ViT) via a novel, effective positional encoding operator. We prove that GE-ViT meets all the theoretical requirements of an equivariant neural network. Comprehensive experiments are conducted on standard benchmark datasets, demonstrating that GE-ViT significantly outperforms non-equivariant self-attention networks. The code is available at https://github.com/ZJUCDSYangKaifan/GEVit.",

author = "Renjun Xu and Kaifan Yang and Ke Liu and Fengxiang He",

year = "2023",

month = jul,

day = "1",

language = "English",

volume = "216",

series = "Proceedings of Machine Learning Research",

publisher = "PMLR",

pages = "2356--2366",

editor = "Evans, {Robin J.} and Ilya Shpitser",

booktitle = "Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence",

note = "39th Conference on Uncertainty in Artificial Intelligence, UAI 2023 ; Conference date: 31-07-2023 Through 04-08-2023",

url = "https://www.auai.org/uai2023/",

}

Xu, R, Yang, K, Liu, K & He, F 2023, Ε(2)-Equivariant Vision Transformer. in RJ Evans & I Shpitser (eds), Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence. vol. 216, Proceedings of Machine Learning Research, PMLR, pp. 2356-2366, 39th Conference on Uncertainty in Artificial Intelligence, Pittsburgh, Pennsylvania, United States, 31/07/23. <https://proceedings.mlr.press/v216/xu23b.html>

Ε(2)-Equivariant Vision Transformer. / Xu, Renjun; Yang, Kaifan; Liu, Ke et al.
Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence. ed. / Robin J. Evans; Ilya Shpitser. Vol. 216 PMLR, 2023. p. 2356-2366 (Proceedings of Machine Learning Research).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Ε(2)-Equivariant Vision Transformer

AU - Xu, Renjun

AU - Yang, Kaifan

AU - Liu, Ke

AU - He, Fengxiang

N1 - Conference code: 39

PY - 2023/7/1

Y1 - 2023/7/1

N2 - Vision Transformer (ViT) has achieved remarkable performance in computer vision. However, positional encoding in ViT makes it substantially difficult to learn the intrinsic equivariance in data. Initial attempts have been made on designing equivariant ViT but are proved defective in some cases in this paper. To address this issue, we design a Group Equivariant Vision Transformer (GE-ViT) via a novel, effective positional encoding operator. We prove that GE-ViT meets all the theoretical requirements of an equivariant neural network. Comprehensive experiments are conducted on standard benchmark datasets, demonstrating that GE-ViT significantly outperforms non-equivariant self-attention networks. The code is available at https://github.com/ZJUCDSYangKaifan/GEVit.

AB - Vision Transformer (ViT) has achieved remarkable performance in computer vision. However, positional encoding in ViT makes it substantially difficult to learn the intrinsic equivariance in data. Initial attempts have been made on designing equivariant ViT but are proved defective in some cases in this paper. To address this issue, we design a Group Equivariant Vision Transformer (GE-ViT) via a novel, effective positional encoding operator. We prove that GE-ViT meets all the theoretical requirements of an equivariant neural network. Comprehensive experiments are conducted on standard benchmark datasets, demonstrating that GE-ViT significantly outperforms non-equivariant self-attention networks. The code is available at https://github.com/ZJUCDSYangKaifan/GEVit.

M3 - Conference contribution

VL - 216

T3 - Proceedings of Machine Learning Research

SP - 2356

EP - 2366

BT - Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence

A2 - Evans, Robin J.

A2 - Shpitser, Ilya

PB - PMLR

T2 - 39th Conference on Uncertainty in Artificial Intelligence

Y2 - 31 July 2023 through 4 August 2023

ER -

Ε(2)-Equivariant Vision Transformer

Abstract

Publication series

Conference

Access to Document

Fingerprint

Cite this