TAC 2020 Tracks
Call for Participation
TAC 2020 Workshop
Call for Participation
Text Analysis Conference (TAC 2020)
Track Evaluations: August - December 2020
Workshop: January 25-26, 2021
U.S. National Institute of Standards and Technology (NIST)
With support from:
U.S. Department of Defense (DOD)
The Text Analysis Conference (TAC) is a series of shared tasks, evaluations and
workshops organized to promote research in Natural Language
Processing and related applications, by providing a large test
collection, common evaluation procedures, and a forum for
organizations to share their results. TAC comprises multiple tracks,
each of which focuses on a particular subproblem of NLP. TAC tracks
aim to improve end-user tasks, but also include diagnostic and
component evaluations situated within the context of end-user tasks.
You are invited to participate in TAC 2020. Organizations may choose to participate in any or all of the TAC tracks. NIST will provide test data for each track, and track participants will run their NLP systems on the data and return their results to NIST for evaluation. The annual conference culminates in a workshop to report on evaluation results.
All results submitted to NIST are archived on the TAC web site, and all evaluations of submitted results are included in the conference proceedings. Dissemination of TAC work and results other than in the conference proceedings is welcomed, but the conditions of participation specifically preclude any advertising claims based on TAC results.
TAC 2020 has three tracks:
Epidemic Question Answering (EPIC-QA)
The goal of the EPIC-QA track is to evaluate systems on their
ability to provide timely and well-supported answers to questions
about the disease COVID-19, its causal virus SARS-CoV-2, related
coronaviruses, and the recommended response to the
pandemic. Because questions arise from both experts and
non-experts in the field, EPIC-QA systems are challenged to return
expert-level answers as expected by the scientific and medical
communities as well as answers in consumer-friendly language for
the general public.
Track coordinators: Dina Demner-Fushman (email@example.com) and Travis Goodwin (firstname.lastname@example.org)
Home page: https://bionlp.nlm.nih.gov/epic_qa/
Group / mailing list: email@example.com
- Recognizing Ultra Fine-grained EntitieS (RUFES)
The goal of the KBP RUFES track is to extract and corefer mentions of fine-grained entity types in text.
Track coordinator: Heng Ji (firstname.lastname@example.org) and Avirup Sil (email@example.com)
Home page: http://tac.nist.gov/2020/KBP/RUFES/
Group / mailing list: firstname.lastname@example.org
- Streaming Multimedia Knowledge Base Population (SM-KBP)
The goal of the SM-KBP track is to develop and evaluate technologies that extract structured Knowledge Elements (KEs) from a variety of unstructured sources in order to generate explicit alternative interpretations of events, situations, and trends in noisy, conflicting, and potentially deceptive information environments.
Track coordinator Hoa Dang (email@example.com)
Home page: https://tac.nist.gov/2020/KBP/SM-KBP/
Group/ mailing list: firstname.lastname@example.org
Organizations wishing to participate in any of the TAC 2020 tracks are
invited to register online by November 1, 2020 at:
Participants are advised to register and submit all required agreement forms as soon as possible in order to receive timely access to track resources, including any sample and training data. Participants should also make sure they are subscribed to the mailing lists for the tracks for which they are registered. Registration for a track does not commit you to participating in the track, but is helpful to know for planning. Late registration will be permitted only if resources allow.
Any questions about conference participation may be sent to the TAC
project manager: email@example.com.
The TAC 2020 workshop will be held online on January 25-26, 2021.
The TAC workshop is a forum both for
presentation of results (including failure analyses and system
comparisons), and for more lengthy system presentations describing
techniques used, experiments run on the data, and other issues of
interest to researchers in NLP.
|TAC 2020 Schedule|
|November 1, 2020||Deadline for registration for track participation|
|August - December 2020||Track evaluation windows (varies by track)|
|By December 2020||Release of individual evaluated results to participants (most tracks)|
|January 15, 2021||Deadline for system reports (workshop notebook version)|
|January 25-26, 2021||TAC 2020 workshop (online)|
|March 1, 2021||Deadline for system reports (final proceedings version)|