Recognizing Ultra Fine-grained Entities (RUFES) 2022-2023

Overview

A goal of TAC KBP is to extract mentions of pre-defined entity types from any language, and cluster together mentions of the same entity. Many real world applications in scenarios such as disaster relief and technical support require systems that recognize a wide variety of entity types (e.g., different types of vehicles, various diseases, and biomedical entities) with limited training data for each type. The KBP RUFES task (Recognizing Ultra Fine-grained EntitieS) challenges systems to recognize name, nominal, and pronominal mentions of entities in news articles, from an ontology with over 300 types that cover a variety of topics in the news. In addition, humans will provide "feedback" on system output, which can be used to improve each system.

In the RUFES 2022-2023 evaluation cycle, participants are invited to submit system output over 3 different evaluation datasets:

1. RUFES 2022 evaluation source corpus (10,000 English documents)
2. RUFES 2023 English evaluation source corpus (10,000 English documents)
3. RUFES 2023 Chinese evaluation source corpus (10,000 Chinese documents)

Organizing Committee

Hoa Trang Dang (U.S. National Institute of Standards and Technology, hoa.dang@nist.gov)
Shudong Huang (U.S. National Institute of Standards and Technology, shudong.huang@nist.gov)
Heng Ji (University of Illinois at Urbana-Champaign, hengji@illinois.edu)
Ian Soboroff (U.S. National Institute of Standards and Technology, ian.soboroff@nist.gov)