TAC 2016 Tracks
Cold Start SF/KB
EDL
Ensembling
Guidelines
Tools
Schedule
Event
Belief/Sentiment
Data
Schedule
Organizers
Track Registration
Reporting Guidelines
TAC 2016 Workshop
TAC KBP 2016 Validation/Ensembling Track
Overview
The slot filler validation (SFV) task of the Validation/Ensembling track focuses on the refinement of output from English slot filling (SF) systems by either combining information from multiple slot filling systems, or applying more intensive linguistic processing to validate individual candidate slot fillers.
SFV is a diagnostic task that eliminates the need for a team to have a full slot filling system and concentrates on the refinement of existing system(s) output. The input to the SFV system is a set of submission files from Cold Start KB and slot filling runs. The output of the SFV system is a binary classification (Correct/Incorrect) of each candidate slot filler in each SF run.
For the SFV Ensemble task, the evaluation considers the SFV run as a single ensemble Cold Start SF run that includes only selected slot fillers from the input Cold Start SF runs; the objective function for the SFV Ensemble task is to maximize the F-score of the ensemble Cold Start run.
Preliminary Schedule
| Preliminary TAC KBP 2016 Schedule | |
|---|---|
| February 29 | Track registration opens |
| July 15 | Deadline for registration for track participation |
| August - October | Track evaluation windows (varies by track) |
| August 1-14 | EDL First Evaluation Window (EDL1) |
| August 15-29 | Cold Start KB/SF Evaluation Window |
| August 22-Sept 7 | Event Argument Extraction and Linking Evaluation Window |
| Sept 1 | Release EDL1 scores to individual participants |
| Sept 12-19 | EDL Evaluation Window 2 (EDL2) |
| Sept 20-Oct 3 | Event Nugget Detection and Coreference Evaluation Window |
| Sept 26-Oct 3 | Slot Filler Validation/Ensembling Evaluation Window |
| Oct 10-17 | Belief and Sentiment Evaluation Window |
| By mid October | Release of individual evaluated results to participants (most tracks) |
| October 10 | Deadline for short system descriptions |
| October 18 | Deadline for workshop presentation proposals |
| October 20 | Notification of acceptance of presentation proposals |
| Nov 1 | Deadline for system reports (workshop notebook version) |
| November 14-15 | TAC 2016 workshop in Gaithersburg, Maryland, USA |
| February 2017 | Deadline for system reports (final proceedings version) |