Go to the NIST home page Go to the TAC Home Page TAC Banner

Return to TAC Homepage

TAC 2023 Tracks
      Mailing List
Call for Participation
Track Registration
Reporting Guidelines
TAC 2023 Workshop

Recognizing Ultra Fine-grained Entities (RUFES) 2022-2023

Conducted by:
U.S. National Institute of Standards and Technology (NIST)

With support from:
U.S. Department of Defense


A goal of TAC KBP is to extract mentions of pre-defined entity types from any language, and cluster together mentions of the same entity. Many real world applications in scenarios such as disaster relief and technical support require systems that recognize a wide variety of entity types (e.g., different types of vehicles, various diseases, and biomedical entities) with limited training data for each type. The KBP RUFES task (Recognizing Ultra Fine-grained EntitieS) challenges systems to recognize name, nominal, and pronominal mentions of entities in news articles, from an ontology with over 300 types that cover a variety of topics in the news. In addition, humans will provide "feedback" on system output, which can be used to improve each system.

In the RUFES 2022-2023 evaluation cycle, participants are invited to submit system output over 3 different evaluation datasets:

  1. RUFES 2022 evaluation source corpus (10,000 English documents)
  2. RUFES 2023 English evaluation source corpus (10,000 English documents)
  3. RUFES 2023 Chinese evaluation source corpus (10,000 Chinese documents)

Organizing Committee

    Hoa Trang Dang (U.S. National Institute of Standards and Technology, hoa.dang@nist.gov)
    Shudong Huang (U.S. National Institute of Standards and Technology, shudong.huang@nist.gov)
    Heng Ji (University of Illinois at Urbana-Champaign, hengji@illinois.edu)
    Ian Soboroff (U.S. National Institute of Standards and Technology, ian.soboroff@nist.gov)

NIST is an agency of the
U.S. Department of Commerce

privacy policy / security notice / accessibility statement

Last updated: Thursday, 28-Sep-2023 16:04:09 EDT
Comments to: tac-web@nist.gov