EventBERT: A Pre-Trained Model for Event Correlation Reasoning

Zhou, Y; Geng, X; Shen, T; Long, G; Jiang, D

EventBERT: A Pre-Trained Model for Event Correlation Reasoning

Zhou, Y Geng, X Shen, T Long, G

Jiang, D

Permalink

Publisher:: Association for Computing Machinery (ACM)
Publication Type:: Conference Proceeding
Citation:: WWW 2022 - Proceedings of the ACM Web Conference 2022, 2022, pp. 850-859
Issue Date:: 2022-04-25

Closed Access

	Filename	Description	Size
	Proceeding ACM conference- a pre-trained model.pdf	Published version	3.2 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Zhou, Y
dc.contributor.author	Geng, X
dc.contributor.author	Shen, T
dc.contributor.author	Long, G https://orcid.org/0000-0003-3740-9515
dc.contributor.author	Jiang, D
dc.date.accessioned	2023-03-14T23:24:50Z
dc.date.available	2023-03-14T23:24:50Z
dc.date.issued	2022-04-25
dc.identifier.citation	WWW 2022 - Proceedings of the ACM Web Conference 2022, 2022, pp. 850-859
dc.identifier.isbn	9781450390965
dc.identifier.uri	http://hdl.handle.net/10453/167334
dc.description.abstract	Event correlation reasoning infers whether a natural language paragraph containing multiple events conforms to human common sense. For example, "Andrew was very drowsy, so he took a long nap, and now he is very alert"is sound and reasonable. In contrast, "Andrew was very drowsy, so he stayed up a long time, now he is very alert"does not comply with human common sense. Such reasoning capability is essential for many downstream tasks, such as script reasoning, abductive reasoning, narrative incoherence, story cloze test, etc. However, conducting event correlation reasoning is challenging due to a lack of large amounts of diverse event-based knowledge and difficulty in capturing correlation among multiple events. In this paper, we propose EventBERT, a pre-trained model to encapsulate eventuality knowledge from unlabeled text. Specifically, we collect a large volume of training examples by identifying natural language paragraphs that describe multiple correlated events and further extracting event spans in an unsupervised manner. We then propose three novel event- and correlation-based learning objectives to pre-train an event correlation model on our created training corpus. Experimental results show EventBERT outperforms strong baselines on four downstream tasks, and achieves state-of-the-art results on most of them. Moreover, it outperforms existing pre-trained models by a large margin, e.g., 6.5 23%, in zero-shot learning of these tasks.
dc.language	en
dc.publisher	Association for Computing Machinery (ACM)
dc.relation.ispartof	WWW 2022 - Proceedings of the ACM Web Conference 2022
dc.relation.ispartof	Proceedings of the ACM Web Conference 2022
dc.relation.isbasedon	10.1145/3485447.3511928
dc.rights	info:eu-repo/semantics/closedAccess
dc.title	EventBERT: A Pre-Trained Model for Event Correlation Reasoning
dc.type	Conference Proceeding
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	closed_access	*
dc.date.updated	2023-03-14T23:24:47Z
pubs.publication-status	Published

Abstract:

Event correlation reasoning infers whether a natural language paragraph containing multiple events conforms to human common sense. For example, "Andrew was very drowsy, so he took a long nap, and now he is very alert"is sound and reasonable. In contrast, "Andrew was very drowsy, so he stayed up a long time, now he is very alert"does not comply with human common sense. Such reasoning capability is essential for many downstream tasks, such as script reasoning, abductive reasoning, narrative incoherence, story cloze test, etc. However, conducting event correlation reasoning is challenging due to a lack of large amounts of diverse event-based knowledge and difficulty in capturing correlation among multiple events. In this paper, we propose EventBERT, a pre-trained model to encapsulate eventuality knowledge from unlabeled text. Specifically, we collect a large volume of training examples by identifying natural language paragraphs that describe multiple correlated events and further extracting event spans in an unsupervised manner. We then propose three novel event- and correlation-based learning objectives to pre-train an event correlation model on our created training corpus. Experimental results show EventBERT outperforms strong baselines on four downstream tasks, and achieves state-of-the-art results on most of them. Moreover, it outperforms existing pre-trained models by a large margin, e.g., 6.5 23%, in zero-shot learning of these tasks.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/167334