TAC 2011 Knowledge Base Population (KBP2011) Track

KBP 2010 slot filling utilities by Ralph Grishman:

KBP 2010 evaluation Submission Validation Tool

CheckSFResponse.java, to validate submission format.

Key File Validation tool

CheckSFKey.java, SFQuery.java and SFQueryList.java make basic checks on the format of a 2010 key file for slot filling.  To run,
javac CheckSFKey.java
java CheckSFKey key-file

Conversion tools (2009 to 2010)

UpdateSFKey (ver 1.0) converts a 2009-format key file to a 2010-format key file.  Slot names are not updated.  To run,
javac UpdateSFKey.java
java UpdateSFKey 2009-key-file 2010-key-file
UpdateSFResponse (ver 1.0) converts a 2009-format response file to a 2010-format response file.  Slot names are not updated.  This is of interest only to KBP old-timers with 2009 response files.
javac UpdateSFResponse.java
java UpdateSFResponse 2009-response-file 2010-response-file

Key File analysis tool

KeyStats (ver. 1.0) displays statistics on the frequency of different slots in a key file.
javac KeyStats.java
java KeyStats key-file

Slot Filling Annotation toolkit by CUNY

An annotation tool by Zheng Chen (CUNY) is available for participant annotation. Please contact Heng Ji <hengji@cs.qc.cuny.edu> to obtain the tool.

Format Validation Tools for KBP 2011:

Validate English entity linking submissions: check_kbp_2011_english-entity-linking.pl.

Validate cross-lingual entity linking submissions: check_kbp_2011_cross-lingual-entity-linking.pl.

Validate slot filling submissions: check_kbp_2011_slot-filling.pl

Validate temporal slot filling submissions: check_kbp_2011_temporal_slot-filling.pl