Mitigating covertly unsafe text within natural language systems

Alex Mei; Anisha Kabir; Sharon Levy; Melanie Subbiah; Emily Allaway; John Judge; Desmond Patton; Bruce Bimber; Kathleen McKeown; William Yang Wang

doi:10.18653/v1/2022.findings-emnlp.211

Mitigating covertly unsafe text within natural language systems

Alex Mei, Anisha Kabir, Sharon Levy, Melanie Subbiah, Emily Allaway, John Judge, Desmond Patton, Bruce Bimber, Kathleen McKeown, William Yang Wang

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

An increasingly prevalent problem for intelligent technologies is text safety, as uncontrolled systems may generate recommendations to their users that lead to injury or life-threatening consequences. However, the degree of explicitness of a generated statement that can cause physical harm varies. In this paper, we distinguish types of text that can lead to physical harm and establish one particularly underexplored category: covertly unsafe text. Then, we further break down this category with respect to the system’s information and discuss solutions to mitigate the generation of text in each of these subcategories. Ultimately, our work defines the problem of covertly unsafe language that causes physical harm and argues that this subtle yet dangerous issue needs to be prioritized by stakeholders and regulators. We highlight mitigation strategies to inspire future researchers to tackle this challenging problem and help improve safety within smart systems.

Original language	English
Title of host publication	Findings of the Association for Computational Linguistics: EMNLP 2022
Editors	Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
Place of Publication	Abu Dhabi, United Arab Emirates
Publisher	Association for Computational Linguistics
Pages	2914–2926
Number of pages	13
Edition	3
ISBN (Electronic)	9781959429432
DOIs	https://doi.org/10.18653/v1/2022.findings-emnlp.211
Publication status	Published - 11 Dec 2022
Event	The 2022 Conference on Empirical Methods in Natural Language Processing - Abu Dhabi National Exhibition Centre, Abu Dhabi, United Arab Emirates Duration: 7 Dec 2022 → 11 Dec 2022 Conference number: 27 https://2022.emnlp.org/

Publication series

Name	Findings of the Association for Computational Linguistics
Publisher	ACL
ISSN (Print)	0891-2017
ISSN (Electronic)	1530-9312

Conference

Conference	The 2022 Conference on Empirical Methods in Natural Language Processing
Abbreviated title	EMNLP 2022
Country/Territory	United Arab Emirates
City	Abu Dhabi
Period	7/12/22 → 11/12/22
Internet address	https://2022.emnlp.org/

Access to Document

10.18653/v1/2022.findings-emnlp.211Licence: Creative Commons: Attribution (CC-BY)

MeiEtalEMNLP2022MitigatingCovertlyUnsafeTextFinal published version, 283 KBLicence: Creative Commons: Attribution (CC-BY)

Cite this

Mei, A., Kabir, A., Levy, S., Subbiah, M., Allaway, E., Judge, J., Patton, D., Bimber, B., McKeown, K., & Wang, W. Y. (2022). Mitigating covertly unsafe text within natural language systems. In Y. Goldberg, Z. Kozareva, & Y. Zhang (Eds.), Findings of the Association for Computational Linguistics: EMNLP 2022 (3 ed., pp. 2914–2926). (Findings of the Association for Computational Linguistics). Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.findings-emnlp.211

Mei, Alex ; Kabir, Anisha ; Levy, Sharon et al. / Mitigating covertly unsafe text within natural language systems. Findings of the Association for Computational Linguistics: EMNLP 2022. editor / Yoav Goldberg ; Zornitsa Kozareva ; Yue Zhang. 3. ed. Abu Dhabi, United Arab Emirates : Association for Computational Linguistics, 2022. pp. 2914–2926 (Findings of the Association for Computational Linguistics).

@inproceedings{19c8ecc7e2a441a1b4f0f30bb6bdaac7,

title = "Mitigating covertly unsafe text within natural language systems",

abstract = "An increasingly prevalent problem for intelligent technologies is text safety, as uncontrolled systems may generate recommendations to their users that lead to injury or life-threatening consequences. However, the degree of explicitness of a generated statement that can cause physical harm varies. In this paper, we distinguish types of text that can lead to physical harm and establish one particularly underexplored category: covertly unsafe text. Then, we further break down this category with respect to the system{\textquoteright}s information and discuss solutions to mitigate the generation of text in each of these subcategories. Ultimately, our work defines the problem of covertly unsafe language that causes physical harm and argues that this subtle yet dangerous issue needs to be prioritized by stakeholders and regulators. We highlight mitigation strategies to inspire future researchers to tackle this challenging problem and help improve safety within smart systems.",

author = "Alex Mei and Anisha Kabir and Sharon Levy and Melanie Subbiah and Emily Allaway and John Judge and Desmond Patton and Bruce Bimber and Kathleen McKeown and Wang, {William Yang}",

year = "2022",

month = dec,

day = "11",

doi = "10.18653/v1/2022.findings-emnlp.211",

language = "English",

series = "Findings of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics",

pages = "2914–2926",

editor = "Yoav Goldberg and Zornitsa Kozareva and Yue Zhang",

booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2022",

edition = "3",

note = "The 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022 ; Conference date: 07-12-2022 Through 11-12-2022",

url = "https://2022.emnlp.org/",

}

Mei, A, Kabir, A, Levy, S, Subbiah, M, Allaway, E, Judge, J, Patton, D, Bimber, B, McKeown, K & Wang, WY 2022, Mitigating covertly unsafe text within natural language systems. in Y Goldberg, Z Kozareva & Y Zhang (eds), Findings of the Association for Computational Linguistics: EMNLP 2022. 3 edn, Findings of the Association for Computational Linguistics, Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, pp. 2914–2926, The 2022 Conference on Empirical Methods in Natural Language Processing, Abu Dhabi, United Arab Emirates, 7/12/22. https://doi.org/10.18653/v1/2022.findings-emnlp.211

Mitigating covertly unsafe text within natural language systems. / Mei, Alex; Kabir, Anisha; Levy, Sharon et al.
Findings of the Association for Computational Linguistics: EMNLP 2022. ed. / Yoav Goldberg; Zornitsa Kozareva; Yue Zhang. 3. ed. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, 2022. p. 2914–2926 (Findings of the Association for Computational Linguistics).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Mitigating covertly unsafe text within natural language systems

AU - Mei, Alex

AU - Kabir, Anisha

AU - Levy, Sharon

AU - Subbiah, Melanie

AU - Allaway, Emily

AU - Judge, John

AU - Patton, Desmond

AU - Bimber, Bruce

AU - McKeown, Kathleen

AU - Wang, William Yang

N1 - Conference code: 27

PY - 2022/12/11

Y1 - 2022/12/11

N2 - An increasingly prevalent problem for intelligent technologies is text safety, as uncontrolled systems may generate recommendations to their users that lead to injury or life-threatening consequences. However, the degree of explicitness of a generated statement that can cause physical harm varies. In this paper, we distinguish types of text that can lead to physical harm and establish one particularly underexplored category: covertly unsafe text. Then, we further break down this category with respect to the system’s information and discuss solutions to mitigate the generation of text in each of these subcategories. Ultimately, our work defines the problem of covertly unsafe language that causes physical harm and argues that this subtle yet dangerous issue needs to be prioritized by stakeholders and regulators. We highlight mitigation strategies to inspire future researchers to tackle this challenging problem and help improve safety within smart systems.

AB - An increasingly prevalent problem for intelligent technologies is text safety, as uncontrolled systems may generate recommendations to their users that lead to injury or life-threatening consequences. However, the degree of explicitness of a generated statement that can cause physical harm varies. In this paper, we distinguish types of text that can lead to physical harm and establish one particularly underexplored category: covertly unsafe text. Then, we further break down this category with respect to the system’s information and discuss solutions to mitigate the generation of text in each of these subcategories. Ultimately, our work defines the problem of covertly unsafe language that causes physical harm and argues that this subtle yet dangerous issue needs to be prioritized by stakeholders and regulators. We highlight mitigation strategies to inspire future researchers to tackle this challenging problem and help improve safety within smart systems.

U2 - 10.18653/v1/2022.findings-emnlp.211

DO - 10.18653/v1/2022.findings-emnlp.211

M3 - Conference contribution

T3 - Findings of the Association for Computational Linguistics

SP - 2914

EP - 2926

BT - Findings of the Association for Computational Linguistics: EMNLP 2022

A2 - Goldberg, Yoav

A2 - Kozareva, Zornitsa

A2 - Zhang, Yue

PB - Association for Computational Linguistics

CY - Abu Dhabi, United Arab Emirates

T2 - The 2022 Conference on Empirical Methods in Natural Language Processing

Y2 - 7 December 2022 through 11 December 2022

ER -

Mei A, Kabir A, Levy S, Subbiah M, Allaway E, Judge J et al. Mitigating covertly unsafe text within natural language systems. In Goldberg Y, Kozareva Z, Zhang Y, editors, Findings of the Association for Computational Linguistics: EMNLP 2022. 3 ed. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics. 2022. p. 2914–2926. (Findings of the Association for Computational Linguistics). doi: 10.18653/v1/2022.findings-emnlp.211

Mitigating covertly unsafe text within natural language systems

Abstract / Description of output

Publication series

Conference

Access to Document

Fingerprint

Cite this