Go to the NIST home page Go to the TAC Home Page TAC Banner

Return to TAC Homepage

TAC 2015 Tracks
    Cold Start KBP
    Tri-Lingual EDL
      Event Nugget
      Event Argument
Track Registration
Reporting Guidelines
TAC 2015 Workshop

TAC KBP 2015 Event Argument Tasks


The two Event Argument (EA) tasks of the TAC KBP Event track focus on extracting event arguments and linking arguments that belong to the same event.

EA Task 1: Event Argument Extraction and Linking

Systems in this task aim to extract information about entities (and times) and the role they play in an event and then link those arguments that participate in the same event. Participating systems will extract tuples that include (EventType, Role, Argument) and group these tuples so that, for example the date, location and thing purchased of the same transaction event appear in the same frame. EventType and Role will be drawn from an externally specified ontology that is based on ACE 2005 and Rich ERE. Arguments will be strings from within a document representing the canonical (most-specific) name or description of the entity.

The input to this task will be unannotated documents. The output will be a set of event argument extraction assertions and links between these assertions indicating participation in the same event. The official metric will measure argument extraction and linking together. Diagnostic metrics will include measures of only event argument extraction quality (F1 as in 2014) and other measures. A simple baseline event argument linking strategy which links all event arguments where the event type/subtype are the same together has been implemented by BBN and will be distributed via github to all participants.

EA Task 2: Event Argument Verification and Linking

This task is designed to allow participants to explore approaches to event argument linking without building their own event argument extraction systems. It also allows participants to explore approaches that improve the precision of extracted event arguments via system combination and/or verification algorithms. The output of this task is identical to that of EA Task 1 (Event Argument Extraction and Linking). Participants in this task will receive unlinked argument extraction output from all EA Task 1 participants.

The input to this task will be the event argument extraction output of all systems that participate in EA Task 1. The output of this task and official metric will be the same as EA Task 1. Participants in this task will have training data in the form of the assessments from the 2014 Event Argument Extraction task, but there will not be training data created specifically connected to the 2015 EA Task 1 submissions. Participants in this task are expected to verify event argument extractions (i.e. remove incorrect event argument extractions), but will not be allowed to add additional event argument extractions.

Preliminary Schedule

    Preliminary TAC KBP 2015 Schedule
    April 15Track registration opens
    Mid AprilInitial track guidelines posted
    June 30Deadline for registration for track participation
    August 17-24Event (Argument Extraction and Linking) evaluation window
    August 31-Sept 7Event (Argument Verification and Linking) evaluation window
    By mid OctoberRelease of individual evaluated results to participants (varies by track)
    October 20Deadline for short system descriptions
    October 20Deadline for workshop presentation proposals
    October 25Notification of acceptance of presentation proposals
    Nov 8Deadline for system reports (workshop notebook version)
    November 16-17TAC 2015 workshop in Gaithersburg, Maryland, USA
    March 5, 2016Deadline for system reports (final proceedings version)


While the ACE 2005 event annotation is being provided to all participants, this task diverges from ACE in some cases. One example of divergence is the addition of correct answers derived through inference/world knowledge. This evaluation will treat as correct some cases that were explicitly excluded in ACE 2005.

Task Coordinator

Event Arguments: Marjorie Freedman (BBN, mfreedma@bbn.com)

NIST is an agency of the
U.S. Department of Commerce

privacy policy / security notice / accessibility statement

Last updated: Tuesday, 28-Mar-2017 11:22:55 EDT
Comments to: tac-web@nist.gov