================================================================================
TAC KBP 2016 ENTITY DISCOVERY AND LINKING EVALUATION RESULTS (SECOND EDL WINDOW)
================================================================================


Team ID:  HITS
Organization:  Heidelberg Institute for Theoretical Studies gGmbH


*************************************************************

Run ID:  HITS1
Did the run access the live Web during the evaluation window:  No
Did the run use the Wikipedia links in the Reference KB: Yes
Did the run use the relation/slot information encoded in the Reference KB: Yes
Did the run use the text descriptions in the nodes of the Reference KB: Yes
Did the run use any of the distributed EDL1 runs (from the first evaluation window): No

ALL English, Chinese, and Spanish documents:
Prec	Recall	F1	Metric
0.733	0.205	0.320	strong_typed_mention_match
0.623	0.174	0.272	strong_typed_all_match
0.734	0.166	0.271	strong_typed_link_match
0.435	0.202	0.276	strong_typed_nil_match
0.808	0.215	0.339	entity_match
0.756	0.211	0.330	mention_ceaf
0.686	0.192	0.300	typed_mention_ceaf
0.620	0.173	0.271	typed_mention_ceaf_plus
0.853	0.116	0.205	b_cubed
0.655	0.099	0.172	b_cubed_plus

ONLY English documents:
Prec	Recall	F1	Metric
0.890	0.675	0.768	strong_mention_match
0.733	0.556	0.632	strong_typed_mention_match
0.623	0.473	0.538	strong_typed_all_match
0.734	0.449	0.557	strong_typed_link_match
0.435	0.556	0.488	strong_typed_nil_match
0.808	0.574	0.671	entity_match
0.756	0.573	0.652	mention_ceaf
0.686	0.520	0.592	typed_mention_ceaf
0.620	0.470	0.534	typed_mention_ceaf_plus
0.853	0.451	0.590	b_cubed
0.655	0.393	0.491	b_cubed_plus

ONLY Chinese documents:
Prec	Recall	F1	Metric
0.000	0.000	0.000	strong_mention_match
0.000	0.000	0.000	strong_typed_mention_match
0.000	0.000	0.000	strong_typed_all_match
0.000	0.000	0.000	strong_typed_link_match
0.000	0.000	0.000	strong_typed_nil_match
0.000	0.000	0.000	entity_match
0.000	0.000	0.000	mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf_plus
0.000	0.000	0.000	b_cubed
0.000	0.000	0.000	b_cubed_plus

ONLY Spanish documents:
Prec	Recall	F1	Metric
0.000	0.000	0.000	strong_mention_match
0.000	0.000	0.000	strong_typed_mention_match
0.000	0.000	0.000	strong_typed_all_match
0.000	0.000	0.000	strong_typed_link_match
0.000	0.000	0.000	strong_typed_nil_match
0.000	0.000	0.000	entity_match
0.000	0.000	0.000	mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf_plus
0.000	0.000	0.000	b_cubed
0.000	0.000	0.000	b_cubed_plus


*************************************************************

Run ID:  HITS2
Did the run access the live Web during the evaluation window:  No
Did the run use the Wikipedia links in the Reference KB: Yes
Did the run use the relation/slot information encoded in the Reference KB: Yes
Did the run use the text descriptions in the nodes of the Reference KB: Yes
Did the run use any of the distributed EDL1 runs (from the first evaluation window): No

ALL English, Chinese, and Spanish documents:
Prec	Recall	F1	Metric
0.733	0.205	0.320	strong_typed_mention_match
0.623	0.174	0.272	strong_typed_all_match
0.734	0.166	0.271	strong_typed_link_match
0.435	0.202	0.276	strong_typed_nil_match
0.808	0.215	0.339	entity_match
0.749	0.209	0.327	mention_ceaf
0.679	0.190	0.297	typed_mention_ceaf
0.620	0.173	0.271	typed_mention_ceaf_plus
0.855	0.115	0.203	b_cubed
0.656	0.098	0.171	b_cubed_plus

ONLY English documents:
Prec	Recall	F1	Metric
0.890	0.675	0.768	strong_mention_match
0.733	0.556	0.632	strong_typed_mention_match
0.623	0.473	0.538	strong_typed_all_match
0.734	0.449	0.557	strong_typed_link_match
0.435	0.556	0.488	strong_typed_nil_match
0.808	0.574	0.671	entity_match
0.749	0.568	0.646	mention_ceaf
0.679	0.515	0.586	typed_mention_ceaf
0.620	0.470	0.534	typed_mention_ceaf_plus
0.855	0.446	0.586	b_cubed
0.656	0.392	0.491	b_cubed_plus

ONLY Chinese documents:
Prec	Recall	F1	Metric
0.000	0.000	0.000	strong_mention_match
0.000	0.000	0.000	strong_typed_mention_match
0.000	0.000	0.000	strong_typed_all_match
0.000	0.000	0.000	strong_typed_link_match
0.000	0.000	0.000	strong_typed_nil_match
0.000	0.000	0.000	entity_match
0.000	0.000	0.000	mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf_plus
0.000	0.000	0.000	b_cubed
0.000	0.000	0.000	b_cubed_plus

ONLY Spanish documents:
Prec	Recall	F1	Metric
0.000	0.000	0.000	strong_mention_match
0.000	0.000	0.000	strong_typed_mention_match
0.000	0.000	0.000	strong_typed_all_match
0.000	0.000	0.000	strong_typed_link_match
0.000	0.000	0.000	strong_typed_nil_match
0.000	0.000	0.000	entity_match
0.000	0.000	0.000	mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf_plus
0.000	0.000	0.000	b_cubed
0.000	0.000	0.000	b_cubed_plus