TAC 2015 Tracks
Cold Start KBP
TAC 2015 Workshop
TAC KBP 2015 Event Argument Tasks
The two Event Argument (EA) tasks of the TAC KBP Event track focus on extracting event arguments and linking arguments that belong to the same event.
EA Task 1: Event Argument Extraction and Linking
Systems in this task aim to extract information about entities (and times) and the role they play in an event and then link those arguments that participate in the same event. Participating systems will extract tuples that include (EventType, Role, Argument) and group these tuples so that, for example the date, location and thing purchased of the same transaction event appear in the same frame. EventType and Role will be drawn from an externally specified ontology that is based on ACE 2005 and Rich ERE. Arguments will be strings from within a document representing the canonical (most-specific) name or description of the entity.
The input to this task will be unannotated documents. The output will be a set of event argument extraction assertions and links between these assertions indicating participation in the same event. The official metric will measure argument extraction and linking together. Diagnostic metrics will include measures of only event argument extraction quality (F1 as in 2014) and other measures. A simple baseline event argument linking strategy which links all event arguments where the event type/subtype are the same together has been implemented by BBN and will be distributed via github to all participants.
EA Task 2: Event Argument Verification and Linking
This task is designed to allow participants to explore approaches to event argument linking without building their own event argument extraction systems. It also allows participants to explore approaches that improve the precision of extracted event arguments via system combination and/or verification algorithms. The output of this task is identical to that of EA Task 1 (Event Argument Extraction and Linking). Participants in this task will receive unlinked argument extraction output from all EA Task 1 participants.
The input to this task will be the event argument extraction output of all systems that participate in EA Task 1. The output of this task and official metric will be the same as EA Task 1. Participants in this task will have training data in the form of the assessments from the 2014 Event Argument Extraction task, but there will not be training data created specifically connected to the 2015 EA Task 1 submissions. Participants in this task are expected to verify event argument extractions (i.e. remove incorrect event argument extractions), but will not be allowed to add additional event argument extractions.
While the ACE 2005 event annotation is being provided to all participants, this task diverges from ACE in some cases. One example of divergence is the addition of correct answers derived through inference/world knowledge. This evaluation will treat as correct some cases that were explicitly excluded in ACE 2005.
Task CoordinatorEvent Arguments: Marjorie Freedman (BBN, email@example.com)
NIST is an agency of the
U.S. Department of Commerce
Last updated: Tuesday, 28-Mar-2017 11:22:55 EDT
Comments to: firstname.lastname@example.org