A transformer-based standardisation system for Scottish Gaelic

Junfan Huang; Beatrice Alex; Michael Bauer; D.S. Jasin; Liang Yuchao; Robert Thomas; Will Lamb

doi:10.21437/SIGUL.2023-23

A transformer-based standardisation system for Scottish Gaelic

Junfan Huang, Beatrice Alex, Michael Bauer, D.S. Jasin, Liang Yuchao, Robert Thomas, Will Lamb^*

^*Corresponding author for this work

School of Literatures, Languages and Cultures

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

The transition from rule-based to neural-based architectures has made it more difficult for low-resource languages like Scottish Gaelic to participate in modern language technologies. The performance of deep-learning approaches correlates with the availability of training data, and low-resource languages have limited data reserves by definition. Historical and non-standard orthographic texts could be used to supplement training data, but manual conversion of these texts is expensive and timeconsuming. This paper describes the development of a neuralbased orthographic standardisation system for Scottish Gaelic and compares it to an earlier rule-based system. The best performance yielded a precision of 93.92, a recall of 92.20 and a word error rate of 11.01. This was obtained using a transformerbased mixed teacher model which was trained with augmented data

Original language	English
Title of host publication	Proceedings of SIGUL 2023
Subtitle of host publication	2nd Annual Meeting of the ELRA/ISCA SIG on Under-resourced Languages
Publisher	ISCA
Pages	108-112
Number of pages	5
DOIs	https://doi.org/10.21437/SIGUL.2023-23
Publication status	Published - 19 Aug 2023

Keywords / Materials (for Non-textual outputs)

Scottish Gaelic
text standardisation
text normalisation
transformer
neural network
Natural Language Processing (NLP)
machine learning

Access to Document

10.21437/SIGUL.2023-23

HuangEtalS2023ATransformerBasedFinal published version, 578 KB

https://sigul-2023.ilc.cnr.it/wp-content/uploads/2023/08/7_Paper.pdf

Cite this

@inproceedings{624ae2b9f1ea4b81b1a4f1d3e4419c0b,

title = "A transformer-based standardisation system for Scottish Gaelic",

abstract = "The transition from rule-based to neural-based architectures has made it more difficult for low-resource languages like Scottish Gaelic to participate in modern language technologies. The performance of deep-learning approaches correlates with the availability of training data, and low-resource languages have limited data reserves by definition. Historical and non-standard orthographic texts could be used to supplement training data, but manual conversion of these texts is expensive and timeconsuming. This paper describes the development of a neuralbased orthographic standardisation system for Scottish Gaelic and compares it to an earlier rule-based system. The best performance yielded a precision of 93.92, a recall of 92.20 and a word error rate of 11.01. This was obtained using a transformerbased mixed teacher model which was trained with augmented data",

keywords = "Scottish Gaelic, text standardisation, text normalisation, transformer, neural network, Natural Language Processing (NLP), machine learning",

author = "Junfan Huang and Beatrice Alex and Michael Bauer and D.S. Jasin and Liang Yuchao and Robert Thomas and Will Lamb",

year = "2023",

month = aug,

day = "19",

doi = "10.21437/SIGUL.2023-23",

language = "English",

pages = "108--112",

booktitle = "Proceedings of SIGUL 2023",

publisher = "ISCA",

}

TY - GEN

T1 - A transformer-based standardisation system for Scottish Gaelic

AU - Huang, Junfan

AU - Alex, Beatrice

AU - Bauer, Michael

AU - Jasin, D.S.

AU - Yuchao, Liang

AU - Thomas, Robert

AU - Lamb, Will

PY - 2023/8/19

Y1 - 2023/8/19

N2 - The transition from rule-based to neural-based architectures has made it more difficult for low-resource languages like Scottish Gaelic to participate in modern language technologies. The performance of deep-learning approaches correlates with the availability of training data, and low-resource languages have limited data reserves by definition. Historical and non-standard orthographic texts could be used to supplement training data, but manual conversion of these texts is expensive and timeconsuming. This paper describes the development of a neuralbased orthographic standardisation system for Scottish Gaelic and compares it to an earlier rule-based system. The best performance yielded a precision of 93.92, a recall of 92.20 and a word error rate of 11.01. This was obtained using a transformerbased mixed teacher model which was trained with augmented data

AB - The transition from rule-based to neural-based architectures has made it more difficult for low-resource languages like Scottish Gaelic to participate in modern language technologies. The performance of deep-learning approaches correlates with the availability of training data, and low-resource languages have limited data reserves by definition. Historical and non-standard orthographic texts could be used to supplement training data, but manual conversion of these texts is expensive and timeconsuming. This paper describes the development of a neuralbased orthographic standardisation system for Scottish Gaelic and compares it to an earlier rule-based system. The best performance yielded a precision of 93.92, a recall of 92.20 and a word error rate of 11.01. This was obtained using a transformerbased mixed teacher model which was trained with augmented data

KW - Scottish Gaelic

KW - text standardisation

KW - text normalisation

KW - transformer

KW - neural network

KW - Natural Language Processing (NLP)

KW - machine learning

UR - https://sigul-2023.ilc.cnr.it/

UR - https://www.isca-archive.org/sigul_2023/

U2 - 10.21437/SIGUL.2023-23

DO - 10.21437/SIGUL.2023-23

M3 - Conference contribution

SP - 108

EP - 112

BT - Proceedings of SIGUL 2023

PB - ISCA

ER -

A transformer-based standardisation system for Scottish Gaelic

Abstract

Keywords / Materials (for Non-textual outputs)

Access to Document

Other files and links

Fingerprint

Cite this