Recognizing Ultra Fine-grained Entities, Events, and Relations (RUFEERS) 2024
Conducted by:
U.S. National Institute of Standards and Technology (NIST)
With support from:
U.S. Department of Defense
Overview
The goal of the RUFEERS track is to extract information about
entities, events, and relations such that the information would be
suitable as input to a knowledge base. Many real world applications
in scenarios such as disaster relief and technical support require
systems that recognize a wide variety of entity, event, and relation types
(e.g., different types of vehicles, various diseases, and biomedical
entities) with limited training data for each type. The RUFEERS
track challenges systems to recognize mentions of entities, events, and
relations in news articles, from an ontology of
approximately 55 event, 30 relation, and 350 entity types that cover a variety
of topics. Participating systems will extract mentions
of entities, events, and relations from the ontology, including the roles (if any) that the entities play in the events and relations.
The track will evaluate systems on three tasks. Given an ontology and a set of Washington Post articles:
- Task 1: Extract one mention of each event, relation, and event/relation argument from each document
- Task 2: Extract all mentions of events, relations, and their arguments from each document
- Task 3: Extract all mentions of each entity from each document
Organizing Committee
Shudong Huang (U.S. National Institute of Standards and Technology, shudong.huang@nist.gov)
Hoa Trang Dang (U.S. National Institute of Standards and Technology, hoa.dang@nist.gov)
|