TAC KBP 2017 Event Track

Overview

The goal of the TAC KBP Event track is to extract information about events such that the information would be suitable as input to a knowledge base. In 2017, there will be document-level evaluations of Event Nugget (EN) Detection, Coreference, and Sequencing tasks, and Event Argument and Linking (EAL) tasks. All Event tasks (except event sequencing) will be in three languages: English, Chinese, and Spanish.

As in 2015-2016, all Event tasks will operate using the Rich ERE definitions of events, nuggets, event hopper coreference, and argument validity. As in 2014-2016, the argument task will continue to support some inference of arguments that were ignored through "trumping" in Rich ERE. The evaluations will use a reduced event taxonomy (the same as in 2016).

The document level tasks will all be scored via alignment with an LDC gold standard conforming to RichERE guidelines. This is the same scoring paradigm as in 2016.

Except for the new event sequencing task, there will not be task specific training data created in 2017. Instead, participants will be encouraged to use training data from previous years (Rich ERE, EN annotation, EA assessments, ACE).

Participants can submit to any number and combination of tasks in the Event track. Because input documents are shared across the tasks, participants must refrain from examining any of the input documents before they have finished submitting results for all of the Event tasks in which they are participating.

References

• ACE 2005 Training Data: http://catalog.ldc.upenn.edu/LDC2006T06

While the ACE 2005 event annotation is being provided to all participants, this task diverges from ACE in some cases. One example of divergence is the addition of correct answers derived through inference/world knowledge. This evaluation will treat as correct some cases that were explicitly excluded in ACE 2005.

Track Coordinators