Knowledge Base Population (KBP) 2014

Overview

The goal of TAC Knowledge Base Population (KBP) is to develop and evaluate technologies for building and populating knowledge bases (KBs) from unstructured text. KBP systems must ultimately build a KB from scratch, but must also be able to populate an existing reference KB that has incomplete or unkown provenance.

The task of Cold Start is to build a KB from scratch, using a predefined KB schema and a collection of unstructured text. The current KB schema consists of named entities that can be a person (PER), organization (ORG), or geopolitical entity (GPE); and predefined attributes (a.k.a "slots") for those named entities.

Cold Start KBP can be decomposed into two complementary tasks: entity linking, in which entity mentions must be linked to entities in a reference KB or new entities discovered in the document collection; and slot filling, which involves finding predefined attributes about target entities in unstructured text. The reference KB for 2014 is based on a snapshot of English Wikipedia from October 2008 and is the same reference KB that has been used in TAC KBP since 2009. Each node in the reference KB corresponds to a Wikipedia page for a person, organization, or geopolitical entity and consists of predefined attributes derived from Wikipedia infoboxes. Unstructured text from the Wikipedia page is also available (as "wiki_text") in the reference KB.

The event and sentiment tracks in KBP 2014 aim to extract information about events and sentiment from unstructured text, such that the information would be suitable as input into a structured KB.

Finally, the slot filler validation track focuses on the refinement of output from slot filling systems by either combining information from multiple slot filling systems, or applying more intensive linguistic processing to validate candidate slot fillers.

Tracks

• Cold Start KBP
  The Cold Start track builds a knowledge base from scratch.
  Track coordinator: James Mayfield (jamesmayfield@gmail.com)
  Track home page: http://tac.nist.gov/2014/KBP/ColdStart/
  


• Entity Linking
  The entity linking task is to discover and link names in a document collection to entities in a reference KB, or to new named entities discovered in the document collection.
  Track coordinator: Heng Ji (jih@rpi.edu)
  Track home page: http://nlp.cs.rpi.edu/kbp/2014/
  


• Slot Filling
  The slot filling task is to search a document collection to fill in values for predefined slots (attributes) for a given entity in a reference KB.
  Track coordinator: Mihai Surdeanu (msurdeanu@email.arizona.edu)
  Track home page: http://surdeanu.info/kbp2014/index.php
  


• Slot Filler Validation
  The Slot Filler Validation track focuses on the refinement of output from slot filling systems by either combining information from multiple slot filling systems, or applying more intensive linguistic processing to validate individual candidate slot fillers.
  Track coordinator: Hoa Dang (hoa.dang@nist.gov) and Yasaman Haghpanah (yasaman.haghpanah@nist.gov)
  Track home page: http://tac.nist.gov/2014/KBP/SFValidation/
  


• Sentiment
  The goal of the Sentiment track is to assess the quality of detectors for scoped and attributed sentiment.
  Track coordinator: Claire Cardie (cardie@cs.cornell.edu)
  Track home page: http://tac.nist.gov/2014/KBP/Sentiment/
  


• Event
  The goal of the Event track is to extract information about events such that the information would be suitable as input to a knowledge base.
  Track coordinator: Marjorie Freedman (mfreedma@bbn.com)
  Track home page: http://tac.nist.gov/2014/KBP/Event/
  

Preliminary Schedule

TAC KBP 2014 Schedule
March	Initial track guidelines posted
April	Distribution of document collections
June 15	Deadline for registration for track participation
July - September	Track evaluation windows (varies by track)
October 7	Deadline for short system descriptions
October 7	Deadline for workshop presentation proposals
By October	Release of individual evaluated results to participants (varies by track)
mid October	Notification of acceptance of presentation proposals
Nov 1 Nov 9	Deadline for system reports (workshop notebook version)
November 17-18	TAC 2014 workshop in Gaithersburg, Maryland, USA
February 15, 2015	Deadline for system reports (final proceedings version)

Organizing Committee (Partial)