KBP 2013 Tracks
Entity Linking
Slot Filling (SF)
Temporal SF
Spanish SF
Sentiment SF
SF Validation
Cold Start
Call for Participation
Track Registration
Schedule
Data
Reporting Guidelines
Workshop
Organizers
Knowledge Base Population (KBP) 2013
Evaluation: February-November, 2013
Workshop: November 18-19, 2013
Follow us on twitter @TACKBP2013 U.S. National Institute of Standards and Technology (NIST)
With support from:
U.S. Department of Defense
Overview
The goal of TAC Knowledge Base Population (KBP) is to develop and evaluate technologies for building and populating knowledge bases (KBs) about named entities from unstructured text. KBP systems must either populate an existing reference KB, or else build a KB from scratch. The reference KB for 2013 is based on a snapshot of English Wikipedia from October 2008 and is the same reference KB that has been used in TAC KBP since 2009. Each node in the reference KB corresponds to a Wikipedia page for a person (PER), organization (ORG), or geopolitical entity (GPE) and consists of predefined attributes (a.k.a "slots") derived from Wikipedia infoboxes. Unstructured text from the Wikipedia page is also available (as "wiki_text") in the reference KB.
KBP can be decomposed into two complementary tracks: entity linking, in which entity mentions must be aligned with entities in the reference KB or new entities discovered in the document collection; and slot filling, which involves finding predefined attributes about target entities in unstructured text. A third major KBP track is Cold Start KBP, which combines entity linking and slot filling to populate an empty KB using the predefined schema from slot filling. Three additional tracks -- temporal slot filling, sentiment slot filling, and Spanish slot filling -- enrich the KB by adding temporal constraints to slot filling relations, augmenting the KB with relations involving sentiment between KBP entities, and filling in the KB using information from both English and Spanish documents. Finally, the slot filler validation track focuses on the refinement of output from slot filling systems by either combining information from multiple slot filling systems, or applying more intensive linguistic processing to validate candidate slot fillers.
Tracks
- Entity Linking
The entity linking task is to link entity mentions in a document collection to entities in a reference KB, or to new named entities discovered in the document collection.
Track coordinator: James Mayfield (james.mayfield@jhuapl.edu)
Track home page: http://tac.nist.gov/2013/KBP/EntityLinking/
- English Slot Filling
The slot filling task is to search a document collection to fill in values for predefined slots (attributes) for a given entity in a reference KB.
Track coordinator: Mihai Surdeanu (msurdeanu@email.arizona.edu)
Track home page: http://surdeanu.info/kbp2013/
- Temporal Slot Filling
The goal of temporal slot filling is to augment a KB with temporal constraints on slot filling relations.
Track coordinator: Mihai Surdeanu (msurdeanu@email.arizona.edu)
Track home page: http://surdeanu.info/kbp2013/
- Cross-lingual Spanish Slot Filling
The Spanish slot filling track extends English slot filling to the cross-lingual paradigm, in which English and Spanish documents are used to populate an English reference KB.
Track coordinator: Jason Duncan (crosslingualsf@gmail.com)
Track home page: http://tac.nist.gov/2013/KBP/SpanishSF/
- Sentiment Slot Filling
The overall goal of the Sentiment Slot Filling track is to assess the quality of detectors for scoped and attributed sentiment.
Track coordinator: Margaret Mitchell (m.mitchell@jhu.edu)
Track home page: http://tac.nist.gov/2013/KBP/SentimentSF/
- Slot Filler Validation
The Slot Filler Validation track focuses on the refinement of output from slot filling systems by either combining information from multiple slot filling systems, or applying more intensive linguistic processing to validate individual candidate slot fillers.
Track coordinator: Hoa Dang (hoa.dang@nist.gov)
Track home page: http://tac.nist.gov/2013/KBP/SFValidation/
- Cold Start KBP
The Cold Start track integrates entity linking and slot filling to build a knowledge base from scratch.
Track coordinator: James Mayfield (james.mayfield@jhuapl.edu)
Track home page: http://tac.nist.gov/2013/KBP/ColdStart/
Schedule
| TAC KBP 2013 Schedule | |
|---|---|
| By June | Track guidelines posted |
| By early June | Distribution of document collections |
| June 15 | Deadline for registration for track participation |
| July - September | Track evaluation windows (varies by track) |
| By late October | Release of individual evaluated results to participants (varies by track) |
| September 30 | Deadline for short system descriptions |
| September 30 | Deadline for workshop presentation proposals |
| mid October | Notification of acceptance of presentation proposals |
| November 11 | Deadline for system reports (workshop notebook version) |
| November 18-19 | TAC KBP 2013 workshop in Gaithersburg, Maryland, USA |
| February 15, 2014 | Deadline for system reports (final proceedings version) |
Organizing Committee
Javier Artiles (Rakuten Institute of Technology, javart@gmail.com)Hoa Trang Dang (NIST, hoa.dang@nist.gov)
Jason Duncan (U.S. Department of Defense, crosslingualsf@gmail.com)
Joe Ellis (LDC, joellis@ldc.upenn.edu)
Kira Griffitt (LDC, kiragrif@ldc.upenn.edu)
Ralph Grishman (New York University, grishman@cs.nyu.edu)
Heng Ji (City University of New York, hengji@cs.qc.cuny.edu)
Xuansong Li (LDC, xuansong@ldc.upenn.edu)
James Mayfield (Johns Hopkins University, james.mayfield@jhuapl.edu)
Paul McNamee (Johns Hopkins University, paul.mcnamee@jhuapl.edu)
Margaret Mitchell (Johns Hopkins University, m.mitchell@jhu.edu)
Boyan Onyshkevych (U.S. Department of Defense, baonysh@tycho.ncsc.mil)
Stephanie Strassel (LDC, strassel@ldc.upenn.edu)
Mihai Surdeanu (University of Arizona, msurdeanu@email.arizona.edu)