Go to the NIST home page Go to the TAC Home Page TAC Banner

Return to TAC Homepage

TAC 2015 Tracks
    Cold Start KBP
    Tri-Lingual EDL
      Event Nugget
      Event Argument
Track Registration
Reporting Guidelines
TAC 2015 Workshop

TAC KBP 2015 Event Track


The goal of the TAC KBP Event track is to extract information about events such that the information would be suitable as input to a knowledge base. The track includes Event Nugget (EN) tasks to detect and link events, and Event Argument (EA) tasks to extract event arguments and link arguments that belong to the same event. In 2015, there are 5 sub-tasks of the TAC KBP Event Track:

  • EN Task 1: Event Nugget Detection: This task aims to identify the explicit mentioning of Events in text for English. Participating systems must identify all relevant Event Mention instances within each sentence. Every instance of a mention of the relevant Event types/subtypes taken from the Rich ERE Annotation guidelines must be identified. In addition, systems must identify three REALIS values (ACTUAL, GENERIC, OTHER), which are also described in the Rich ERE guidelines.

  • EN Task 2: Event Nugget Detection and Coreference: In addition to the Event Nugget Detection task described in the EN Task 1, this task also aims to identify Full Event Coreference links at the same time. Full Event Coreference is identified when two or more Event Nuggets refer to the same event. This notion is described as Event Hoppers in the Rich ERE Annotation Guidelines. The Full Event Coreference links do not include subevents relations.

  • EN Task 3: Event Nugget Coreference: This task is to identify Full Event Coreference links, given the annotated Event Nuggets in the text.

  • EA Task 1: Event Argument Extraction and Linking: Systems in this task aim to extract information about entities (and times) and the role they play in an event and then link those arguments that participate in the same event. Participating systems will extract tuples that include (EventType, Role, Argument) and group these tuples so that, for example the date, location and thing purchased of the same transaction event appear in the same frame. EventType and Role will be drawn from an externally specified ontology that is based on ACE 2005 and Rich ERE. Arguments will be strings from within a document representing the canonical (most-specific) name or description of the entity.

  • EA Task 2: Event Argument Verification and Linking: This task is designed to allow participants to explore approaches to event argument linking without building their own event argument extraction systems. It also allows participants to explore approaches that improve the precision of extracted event arguments via system combination and/or verification algorithms. The output of this task is identical to that of EA Task 1 (Event Argument Extraction and Linking). Participants in this task will receive unlinked argument extraction output from all EA Task 1 participants.

For all 5 subtasks of the Event Track, systems will operate over a 100-500 document English corpus that is a mix of newswire and discussion forum documents. A discussion forum document may contain multiple posts. The corpus will be manually and automatically filtered to ensure at least a few instances of all event-types. The discussion-forum posts will be automatically filtered to identify those posts that are not simply reposts of newswire documents. Very long discussion-forum threads will be truncated.

The input documents for the EN tasks will be a subset of the input documents for EA.

Participants can submit to any number and combination of the 5 tasks in the Event track. Because input documents are shared across the 5 tasks, participants must refrain from examining any of the input documents before they have finished submitting results for all of the Event tasks in which they are participating.

Preliminary Schedule

    Preliminary TAC KBP 2015 Schedule
    April 15Track registration opens
    Mid AprilInitial track guidelines posted
    June 30Deadline for registration for track participation
    August 17-24Event (Argument Extraction and Linking) evaluation window
    August 31-Sept 7Event (Argument Verification and Linking) evaluation window
    Sept 8-21Event (Nugget Detection) evaluation window
    Sept 8-21Event (Nugget Detection and Coreference) evaluation window
    Sept 22-29Event (Nugget Coreference) evaluation window
    By mid OctoberRelease of individual evaluated results to participants (varies by track)
    October 20Deadline for short system descriptions
    October 20Deadline for workshop presentation proposals
    October 25Notification of acceptance of presentation proposals
    Nov 8Deadline for system reports (workshop notebook version)
    November 16-17TAC 2015 workshop in Gaithersburg, Maryland, USA
    March 5, 2016Deadline for system reports (final proceedings version)


While the ACE 2005 event annotation is being provided to all participants, this task diverges from ACE in some cases. One example of divergence is the addition of correct answers derived through inference/world knowledge. This evaluation will treat as correct some cases that were explicitly excluded in ACE 2005.

Track Coordinators

Event Arguments: Marjorie Freedman (BBN, mfreedma@bbn.com)
Event Nuggets: Teruko Mitamura (CMU, teruko@cs.cmu.edu) and Eduard Hovy (CMU, ehovy@andrew.cmu.edu)

NIST is an agency of the
U.S. Department of Commerce

privacy policy / security notice / accessibility statement

Last updated: Tuesday, 28-Mar-2017 11:22:18 EDT
Comments to: tac-web@nist.gov