Go to the NIST home page Go to the TAC Home Page TAC Banner

Return to TAC Homepage

TAC 2017 Tracks
  KBP Tracks
      Cold Start KB/SF
      EDL
      Event
        Event Nugget
        Event Argument
      BeSt
    Data
    Schedule
    Organizers
  ADR Track
Call for Participation
Track Registration
Reporting Guidelines
TAC 2017 Workshop




TAC KBP 2017 Event Track

Overview

The goal of the TAC KBP Event track is to extract information about events such that the information would be suitable as input to a knowledge base. In 2017, there will be document-level evaluations of Event Nugget (EN) Detection, Coreference, and Sequencing tasks, and Event Argument and Linking (EAL) tasks. All Event tasks (except event sequencing) will be in three languages: English, Chinese, and Spanish.

As in 2015-2016, all Event tasks will operate using the Rich ERE definitions of events, nuggets, event hopper coreference, and argument validity. As in 2014-2016, the argument task will continue to support some inference of arguments that were ignored through "trumping" in Rich ERE. The evaluations will use a reduced event taxonomy (the same as in 2016).

The document level tasks will all be scored via alignment with an LDC gold standard conforming to RichERE guidelines. This is the same scoring paradigm as in 2016.

Except for the new event sequencing task, there will not be task specific training data created in 2017. Instead, participants will be encouraged to use training data from previous years (Rich ERE, EN annotation, EA assessments, ACE).

Participants can submit to any number and combination of tasks in the Event track. Because input documents are shared across the tasks, participants must refrain from examining any of the input documents before they have finished submitting results for all of the Event tasks in which they are participating.

References

While the ACE 2005 event annotation is being provided to all participants, this task diverges from ACE in some cases. One example of divergence is the addition of correct answers derived through inference/world knowledge. This evaluation will treat as correct some cases that were explicitly excluded in ACE 2005.

Track Coordinators

Event Arguments: Marjorie Freedman (BBN, mfreedma@bbn.com)
Event Nuggets: Teruko Mitamura (CMU, teruko@cs.cmu.edu) and Eduard Hovy (CMU, ehovy@andrew.cmu.edu)


NIST is an agency of the
U.S. Department of Commerce

privacy policy / security notice / accessibility statement
disclaimer
FOIA

Last updated: Tuesday, 28-Mar-2017 11:22:20 EDT
Comments to: tac-web@nist.gov