TAC 2017 Tracks
Call for Participation
TAC 2017 Workshop
Call for Participation
Text Analysis Conference (TAC 2017)
Track Evaluations: February - October 2017
Workshop: November 13-14, 2017
National Institute of Standards and Technology (NIST)
With support from:
U.S. Department of Defense
The Text Analysis Conference (TAC) is a series of shared tasks, evaluations and
workshops organized to promote research in Natural Language
Processing and related applications, by providing a large test
collection, common evaluation procedures, and a forum for
organizations to share their results. TAC comprises multiple tracks,
each of which focuses on a particular subproblem of NLP. TAC tracks
aim to improve end-user tasks, but also include diagnostic and
component evaluations situated within the context of end-user tasks.
TAC 2017 has six tracks in two reseach areas:
- Knowledge Base Population (KBP)
- Adverse Drug Reaction Extraction from Drug Labels (ADR)
You are invited to participate in TAC 2017. Organizations may choose to participate in any or all of the TAC tracks. NIST will provide test data for each track, and track participants will run their NLP systems on the data and return their results to NIST for evaluation. The annual conference culminates in a November workshop at NIST in Gaithersburg, Maryland, USA.
All results submitted to NIST are archived on the TAC web site, and all evaluations of submitted results are included in the conference proceedings. Dissemination of TAC work and results other than in the conference proceedings is welcomed, but the conditions of participation specifically preclude any advertising claims based on TAC results.
TAC 2017 has six tracks in two major areas:
Drug Reaction Extraction from Drug Labels (ADR)
purpose of this track is to test various natural language
approaches for their information extraction performance on adverse
drug reactions (ADR). Participants will be provided with
mention-, relation-, and document-level annotations in order to
extract structured ADR information from the Food and Drug
Administration's (FDA) official pharmaceutical knowledge
Track coordinator: Kirk Roberts (Kirk.Roberts@uth.tmc.edu)
Home page: https://bionlp.nlm.nih.gov/tac2017adversereactions/
Mailing list: firstname.lastname@example.org
- Knowledge Base Population
KBP tracks develop technologies for building and
populating knowledge bases (KBs) from unstructured text. In
addition to the main end-to-end KB construction task in the Cold
Start KB track, component tasks are offered in 4 tracks that
focus on specific components of the KB:
Home page: https://tac.nist.gov/2017/KBP/
- Cold Start KB (CSKB): The Cold Start KB track builds a
knowledge base from scratch using a given document collection and
knowledge base schema. The KB schema includes entities, events, and
relations involving entities and events, including entity attributes
(aka slots), event arguments, and sentiment between entities.
- Entity Discovery and Linking (EDL): The Entity Discovery
and Linking track aims to extract entity mentions from a source
collection of textual documents and link them to a reference KB; an
EDL system is also required to cluster mentions for those entities
that don't have corresponding KB entries.
- Slot Filling (SF): The Slot Filling task is to search a
document collection to fill in values for predefined slots
(attributes) for a given entity.
- Event: The goal of the Event track is to extract
information about events such that the information would be suitable
as input to a knowledge base. The track includes Event Nugget
(EN) tasks to detect and link events, and Event Argument
(EAL) tasks to extract event arguments and link arguments that
belong to the same event.
- Belief and Sentiment (BeSt): The Belief and Sentiment track
detects belief and sentiment of an entity toward another entity,
relation, or event.
Mailing list: email@example.com
Organizations wishing to participate in any of the TAC 2017 tracks are
invited to register online by June 15, 2017 at:
Participants are advised to register and submit all required agreement forms as soon as possible in order to receive timely access to track resources, including any sample and training data. Participants should also make sure they are subscribed to the mailing lists for the tracks for which they are registered. Registration for a track does not commit you to participating in the track, but is helpful to know for planning. Late registration will be permitted only if resources allow.
Any questions about conference participation may be sent to the TAC
project manager: firstname.lastname@example.org.
The TAC 2017 workshop will be held November 13-14, 2017, in
Gaithersburg, Maryland, USA. The TAC workshop is a forum both for
presentation of results (including failure analyses and system
comparisons), and for more lengthy system presentations describing
techniques used, experiments run on the data, and other issues of
interest to researchers in NLP.
|TAC 2017 Schedule|
|February 20||Track registration opens|
|June 15||Deadline for registration for track participation|
|June - October||Track evaluation windows (varies by track)|
|By mid October||Release of individual evaluated results to participants (most tracks)|
|October 15||Deadline for short system descriptions|
|October 15||Deadline for workshop presentation proposals|
|October 20||Notification of acceptance of presentation proposals|
|November 1||Deadline for system reports (workshop notebook version)|
|November 13-14||TAC 2017 workshop in Gaithersburg, Maryland, USA|
|February 2018||Deadline for system reports (final proceedings version)|