Extending defoe for the efficient analysis of historical texts at scale

Rosa Filgueira; Claire Grover; Vasileios Karaiskos; Beatrice Alex; Sarah Van Eyndhoven; Lisa Gotthard; Melissa Terras

doi:10.1109/eScience51609.2021.00012

Extending defoe for the efficient analysis of historical texts at scale

Rosa Filgueira, Claire Grover, Vasileios Karaiskos, Beatrice Alex, Sarah Van Eyndhoven, Lisa Gotthard, Melissa Terras

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

This paper presents the new facilities provided in defoe, a parallel toolbox for querying a wealth of digitised newspapers and books at scale. defoe has been extended to work with further Natural Language Processing () tools such as the Edinburgh Geoparser, to store the preprocessed text in several storage facilities and to support different types of queries and analyses. We have also extended the collection of XML schemas supported by defoe, increasing the versatility of the tool for the analysis of digital historical textual data at scale. Finally, we have conducted several studies in which we worked with humanities and social science researchers who posed complex and interested questions to large-scale digital collections. Results shows that defoe allows researchers to conduct their studies and obtain results faster, while all the large-scale text mining complexity
is automatically handled by defoe.

Original language	English
Title of host publication	2021 IEEE 17th International Conference on eScience (eScience)
Publisher	Institute of Electrical and Electronics Engineers
Pages	21-29
Number of pages	9
ISBN (Electronic)	9781665403610
ISBN (Print)	9781665447089
DOIs	https://doi.org/10.1109/eScience51609.2021.00012
Publication status	Published - 26 Oct 2021
Event	IEEE eScience 2021 - 17th IEEE eScience 2021 International Conference - University of Innsbruck , Innsbruck, Austria Duration: 20 Sept 2021 → 23 Sept 2021 https://www.escience2021.org

Conference

Conference	IEEE eScience 2021 - 17th IEEE eScience 2021 International Conference
Abbreviated title	eScience 2021
Country/Territory	Austria
City	Innsbruck
Period	20/09/21 → 23/09/21
Internet address	https://www.escience2021.org

Keywords / Materials (for Non-textual outputs)

text mining
distributed queries
High Performance Computing
XML schemas
digital tools
humanities research

Access to Document

10.1109/eScience51609.2021.00012

FilgueiraEtal2021ExtendingDefoe
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Accepted author manuscript, 7.6 MB

'Scots for the masses'? Exploring the use of Scots in 19th century chapbooks
Sarah Van Eyndhoven (Speaker), Lisa Gotthard (Speaker) & Rosa Filgueira (Contributor)
Jun 2021
Activity: Academic talk or presentation types › Oral presentation

Cite this

@inproceedings{786e17eb77aa4dc7b32033d17cd7b316,

title = "Extending defoe for the efficient analysis of historical texts at scale",

abstract = "This paper presents the new facilities provided in defoe, a parallel toolbox for querying a wealth of digitised newspapers and books at scale. defoe has been extended to work with further Natural Language Processing () tools such as the Edinburgh Geoparser, to store the preprocessed text in several storage facilities and to support different types of queries and analyses. We have also extended the collection of XML schemas supported by defoe, increasing the versatility of the tool for the analysis of digital historical textual data at scale. Finally, we have conducted several studies in which we worked with humanities and social science researchers who posed complex and interested questions to large-scale digital collections. Results shows that defoe allows researchers to conduct their studies and obtain results faster, while all the large-scale text mining complexityis automatically handled by defoe.",

keywords = "text mining, distributed queries, High Performance Computing, XML schemas, digital tools, humanities research",

author = "Rosa Filgueira and Claire Grover and Vasileios Karaiskos and Beatrice Alex and {Van Eyndhoven}, Sarah and Lisa Gotthard and Melissa Terras",

year = "2021",

month = oct,

day = "26",

doi = "10.1109/eScience51609.2021.00012",

language = "English",

isbn = "9781665447089",

pages = "21--29",

booktitle = "2021 IEEE 17th International Conference on eScience (eScience)",

publisher = "Institute of Electrical and Electronics Engineers",

note = "IEEE eScience 2021 - 17th IEEE eScience 2021 International Conference, eScience 2021 ; Conference date: 20-09-2021 Through 23-09-2021",

url = "https://www.escience2021.org",

}

Filgueira, R , Grover, C, Karaiskos, V, Alex, B, Van Eyndhoven, S, Gotthard, L & Terras, M 2021, Extending defoe for the efficient analysis of historical texts at scale. in 2021 IEEE 17th International Conference on eScience (eScience). Institute of Electrical and Electronics Engineers, pp. 21-29, IEEE eScience 2021 - 17th IEEE eScience 2021 International Conference, Innsbruck, Austria, 20/09/21. https://doi.org/10.1109/eScience51609.2021.00012

TY - GEN

T1 - Extending defoe for the efficient analysis of historical texts at scale

AU - Filgueira, Rosa

AU - Grover, Claire

AU - Karaiskos, Vasileios

AU - Alex, Beatrice

AU - Van Eyndhoven, Sarah

AU - Gotthard, Lisa

AU - Terras, Melissa

PY - 2021/10/26

Y1 - 2021/10/26

N2 - This paper presents the new facilities provided in defoe, a parallel toolbox for querying a wealth of digitised newspapers and books at scale. defoe has been extended to work with further Natural Language Processing () tools such as the Edinburgh Geoparser, to store the preprocessed text in several storage facilities and to support different types of queries and analyses. We have also extended the collection of XML schemas supported by defoe, increasing the versatility of the tool for the analysis of digital historical textual data at scale. Finally, we have conducted several studies in which we worked with humanities and social science researchers who posed complex and interested questions to large-scale digital collections. Results shows that defoe allows researchers to conduct their studies and obtain results faster, while all the large-scale text mining complexityis automatically handled by defoe.

AB - This paper presents the new facilities provided in defoe, a parallel toolbox for querying a wealth of digitised newspapers and books at scale. defoe has been extended to work with further Natural Language Processing () tools such as the Edinburgh Geoparser, to store the preprocessed text in several storage facilities and to support different types of queries and analyses. We have also extended the collection of XML schemas supported by defoe, increasing the versatility of the tool for the analysis of digital historical textual data at scale. Finally, we have conducted several studies in which we worked with humanities and social science researchers who posed complex and interested questions to large-scale digital collections. Results shows that defoe allows researchers to conduct their studies and obtain results faster, while all the large-scale text mining complexityis automatically handled by defoe.

KW - text mining

KW - distributed queries

KW - High Performance Computing

KW - XML schemas

KW - digital tools

KW - humanities research

U2 - 10.1109/eScience51609.2021.00012

DO - 10.1109/eScience51609.2021.00012

M3 - Conference contribution

SN - 9781665447089

SP - 21

EP - 29

BT - 2021 IEEE 17th International Conference on eScience (eScience)

PB - Institute of Electrical and Electronics Engineers

T2 - IEEE eScience 2021 - 17th IEEE eScience 2021 International Conference

Y2 - 20 September 2021 through 23 September 2021

ER -

Extending defoe for the efficient analysis of historical texts at scale

Abstract / Description of output

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Fingerprint

Activities

'Scots for the masses'? Exploring the use of Scots in 19th century chapbooks

Cite this