Go to the NIST home page Go to the TAC Home Page TAC Banner

Return to TAC Homepage

KBP 2013 Tracks
  Entity Linking
  Slot Filling (SF)
  Temporal SF
  Spanish SF
  Sentiment SF
  SF Validation
  Cold Start

Call for Participation
Track Registration
Reporting Guidelines

Knowledge Base Population (KBP) 2013

Evaluation: February-November, 2013
Workshop: November 18-19, 2013

Follow us on twitter @TACKBP2013

Conducted by:
U.S. National Institute of Standards and Technology (NIST)

With support from:
U.S. Department of Defense


The goal of TAC Knowledge Base Population (KBP) is to develop and evaluate technologies for building and populating knowledge bases (KBs) about named entities from unstructured text. KBP systems must either populate an existing reference KB, or else build a KB from scratch. The reference KB for 2013 is based on a snapshot of English Wikipedia from October 2008 and is the same reference KB that has been used in TAC KBP since 2009. Each node in the reference KB corresponds to a Wikipedia page for a person (PER), organization (ORG), or geopolitical entity (GPE) and consists of predefined attributes (a.k.a "slots") derived from Wikipedia infoboxes. Unstructured text from the Wikipedia page is also available (as "wiki_text") in the reference KB.

KBP can be decomposed into two complementary tracks: entity linking, in which entity mentions must be aligned with entities in the reference KB or new entities discovered in the document collection; and slot filling, which involves finding predefined attributes about target entities in unstructured text. A third major KBP track is Cold Start KBP, which combines entity linking and slot filling to populate an empty KB using the predefined schema from slot filling. Three additional tracks -- temporal slot filling, sentiment slot filling, and Spanish slot filling -- enrich the KB by adding temporal constraints to slot filling relations, augmenting the KB with relations involving sentiment between KBP entities, and filling in the KB using information from both English and Spanish documents. Finally, the slot filler validation track focuses on the refinement of output from slot filling systems by either combining information from multiple slot filling systems, or applying more intensive linguistic processing to validate candidate slot fillers.


  • Entity Linking
    The entity linking task is to link entity mentions in a document collection to entities in a reference KB, or to new named entities discovered in the document collection.
    Track coordinator: James Mayfield (james.mayfield@jhuapl.edu)
    Track home page: http://tac.nist.gov/2013/KBP/EntityLinking/

  • English Slot Filling
    The slot filling task is to search a document collection to fill in values for predefined slots (attributes) for a given entity in a reference KB.
    Track coordinator: Mihai Surdeanu (msurdeanu@email.arizona.edu)
    Track home page: http://surdeanu.info/kbp2013/

  • Temporal Slot Filling
    The goal of temporal slot filling is to augment a KB with temporal constraints on slot filling relations.
    Track coordinator: Mihai Surdeanu (msurdeanu@email.arizona.edu)
    Track home page: http://surdeanu.info/kbp2013/

  • Cross-lingual Spanish Slot Filling
    The Spanish slot filling track extends English slot filling to the cross-lingual paradigm, in which English and Spanish documents are used to populate an English reference KB.
    Track coordinator: Jason Duncan (crosslingualsf@gmail.com)
    Track home page: http://tac.nist.gov/2013/KBP/SpanishSF/

  • Sentiment Slot Filling
    The overall goal of the Sentiment Slot Filling track is to assess the quality of detectors for scoped and attributed sentiment.
    Track coordinator: Margaret Mitchell (m.mitchell@jhu.edu)
    Track home page: http://tac.nist.gov/2013/KBP/SentimentSF/

  • Slot Filler Validation
    The Slot Filler Validation track focuses on the refinement of output from slot filling systems by either combining information from multiple slot filling systems, or applying more intensive linguistic processing to validate individual candidate slot fillers.
    Track coordinator: Hoa Dang (hoa.dang@nist.gov)
    Track home page: http://tac.nist.gov/2013/KBP/SFValidation/

  • Cold Start KBP
    The Cold Start track integrates entity linking and slot filling to build a knowledge base from scratch.
    Track coordinator: James Mayfield (james.mayfield@jhuapl.edu)
    Track home page: http://tac.nist.gov/2013/KBP/ColdStart/


    TAC KBP 2013 Schedule
    By JuneTrack guidelines posted
    By early JuneDistribution of document collections
    June 15Deadline for registration for track participation
    July - SeptemberTrack evaluation windows (varies by track)
    By late OctoberRelease of individual evaluated results to participants (varies by track)
    September 30Deadline for short system descriptions
    September 30Deadline for workshop presentation proposals
    mid OctoberNotification of acceptance of presentation proposals
    November 11Deadline for system reports (workshop notebook version)
    November 18-19TAC KBP 2013 workshop in Gaithersburg, Maryland, USA
    February 15, 2014Deadline for system reports (final proceedings version)

Organizing Committee

Javier Artiles (Rakuten Institute of Technology, javart@gmail.com)
Hoa Trang Dang (NIST, hoa.dang@nist.gov)
Jason Duncan (U.S. Department of Defense, crosslingualsf@gmail.com)
Joe Ellis (LDC, joellis@ldc.upenn.edu)
Kira Griffitt (LDC, kiragrif@ldc.upenn.edu)
Ralph Grishman (New York University, grishman@cs.nyu.edu)
Heng Ji (City University of New York, hengji@cs.qc.cuny.edu)
Xuansong Li (LDC, xuansong@ldc.upenn.edu)
James Mayfield (Johns Hopkins University, james.mayfield@jhuapl.edu)
Paul McNamee (Johns Hopkins University, paul.mcnamee@jhuapl.edu)
Margaret Mitchell (Johns Hopkins University, m.mitchell@jhu.edu)
Boyan Onyshkevych (U.S. Department of Defense, baonysh@tycho.ncsc.mil)
Stephanie Strassel (LDC, strassel@ldc.upenn.edu)
Mihai Surdeanu (University of Arizona, msurdeanu@email.arizona.edu)

NIST is an agency of the
U.S. Department of Commerce

privacy policy / security notice / accessibility statement

Last updated: Tuesday, 28-Mar-2017 11:20:58 EDT
Comments to: tac-web@nist.gov