TAC KBP 2014 Sentiment Track

Overview

The goal of the Sentiment track is to assess the quality of detectors for scoped and attributed sentiment. This track will supplement existing Entity Linking, Slot Filling, and Cold Start tasks with information about opinions held by people and other entities.

The sentiment slot filling task is for systems to find all cases where the query entity (sentiment holder) is documented as holding a positive or negative sentiment about another entity (sentiment target). The sentiment slot filling task is a special case of slot filling in which the slots indicate that the query entity has expressed either a positive or negative sentiment toward the entity that is the slot filler; alternatively, the slots may indicate that the query entity is the target of either a positive or negative sentiment expressed by the slot filler. The slot names in 2014 are limited to the following four sentiment types:

• pos-towards: query entity holds positive sentiment towards filler entity ("likes", "is hopeful about", etc.).
• pos-from: query entity is target of positive sentiment from filler entity ("is liked by", "was hoped for by", etc.).
• neg-towards: query entity holds negative sentiment towards filler entity ("dislikes", "is skeptical about", etc.).
• neg-from: query entity is target of negative sentiment from filler entity ("is disliked by", etc.).

The document collection is expected to be primarily discussion forum posts, although blogs and news articles may also be included. Documents will be formatted in the same way as the documents for the other Knowledge Base Population tracks.

Preliminary Schedule

April 15	Track guidelines posted
April	Distribution of document collections
June 15	Deadline for registration for track participation
July 28 - August 11	Sentiment evaluation window
October	Release of individual evaluated results to participants
October 7	Deadline for short system descriptions
October 7	Deadline for workshop presentation proposals
mid October	Notification of acceptance of presentation proposals
Nov 1 Nov 9	Deadline for system reports (workshop notebook version)
November 17-18	TAC 2014 workshop in Gaithersburg, Maryland, USA
February 15, 2015	Deadline for system reports (final proceedings version)

Track Coordinator