===============================================================================
TAC KBP 2016 ENTITY DISCOVERY AND LINKING EVALUATION RESULTS (FIRST EDL WINDOW)
===============================================================================


Team ID:  summa
Organization:  University College London


*************************************************************

Run ID:  summa1
Did the run access the live Web during the evaluation window:  No
Did the run use the Wikipedia links in the Reference KB: Yes
Did the run use the relation/slot information encoded in the Reference KB: No
Did the run use the text descriptions in the nodes of the Reference KB: No

ALL English, Chinese, and Spanish documents:
Prec	Recall	F1	Metric
0.763	0.144	0.243	strong_typed_mention_match
0.595	0.113	0.189	strong_typed_all_match
0.673	0.136	0.227	strong_typed_link_match
0.210	0.030	0.052	strong_typed_nil_match
0.704	0.200	0.311	entity_match
0.700	0.132	0.223	mention_ceaf
0.643	0.122	0.204	typed_mention_ceaf
0.591	0.112	0.188	typed_mention_ceaf_plus
0.754	0.051	0.095	b_cubed
0.585	0.043	0.079	b_cubed_plus

ONLY English documents:
Prec	Recall	F1	Metric
0.849	0.435	0.575	strong_mention_match
0.763	0.391	0.517	strong_typed_mention_match
0.595	0.305	0.404	strong_typed_all_match
0.673	0.368	0.476	strong_typed_link_match
0.210	0.083	0.119	strong_typed_nil_match
0.704	0.534	0.608	entity_match
0.700	0.359	0.475	mention_ceaf
0.643	0.330	0.436	typed_mention_ceaf
0.591	0.303	0.401	typed_mention_ceaf_plus
0.754	0.241	0.365	b_cubed
0.585	0.209	0.308	b_cubed_plus

ONLY Chinese documents:
Prec	Recall	F1	Metric
0.000	0.000	0.000	strong_mention_match
0.000	0.000	0.000	strong_typed_mention_match
0.000	0.000	0.000	strong_typed_all_match
0.000	0.000	0.000	strong_typed_link_match
0.000	0.000	0.000	strong_typed_nil_match
0.000	0.000	0.000	entity_match
0.000	0.000	0.000	mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf_plus
0.000	0.000	0.000	b_cubed
0.000	0.000	0.000	b_cubed_plus

ONLY Spanish documents:
Prec	Recall	F1	Metric
0.000	0.000	0.000	strong_mention_match
0.000	0.000	0.000	strong_typed_mention_match
0.000	0.000	0.000	strong_typed_all_match
0.000	0.000	0.000	strong_typed_link_match
0.000	0.000	0.000	strong_typed_nil_match
0.000	0.000	0.000	entity_match
0.000	0.000	0.000	mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf_plus
0.000	0.000	0.000	b_cubed
0.000	0.000	0.000	b_cubed_plus


*************************************************************

Run ID:  summa2
Did the run access the live Web during the evaluation window:  No
Did the run use the Wikipedia links in the Reference KB: Yes
Did the run use the relation/slot information encoded in the Reference KB: No
Did the run use the text descriptions in the nodes of the Reference KB: No

ALL English, Chinese, and Spanish documents:
Prec	Recall	F1	Metric
0.798	0.181	0.295	strong_typed_mention_match
0.710	0.161	0.262	strong_typed_all_match
0.696	0.151	0.248	strong_typed_link_match
0.750	0.195	0.309	strong_typed_nil_match
0.738	0.215	0.333	entity_match
0.811	0.184	0.300	mention_ceaf
0.751	0.170	0.277	typed_mention_ceaf
0.706	0.160	0.261	typed_mention_ceaf_plus
0.814	0.093	0.167	b_cubed
0.721	0.085	0.151	b_cubed_plus

ONLY English documents:
Prec	Recall	F1	Metric
0.873	0.537	0.665	strong_mention_match
0.798	0.491	0.608	strong_typed_mention_match
0.710	0.436	0.540	strong_typed_all_match
0.696	0.408	0.514	strong_typed_link_match
0.750	0.537	0.626	strong_typed_nil_match
0.738	0.576	0.647	entity_match
0.811	0.499	0.618	mention_ceaf
0.751	0.462	0.572	typed_mention_ceaf
0.706	0.434	0.537	typed_mention_ceaf_plus
0.814	0.372	0.511	b_cubed
0.721	0.343	0.465	b_cubed_plus

ONLY Chinese documents:
Prec	Recall	F1	Metric
0.000	0.000	0.000	strong_mention_match
0.000	0.000	0.000	strong_typed_mention_match
0.000	0.000	0.000	strong_typed_all_match
0.000	0.000	0.000	strong_typed_link_match
0.000	0.000	0.000	strong_typed_nil_match
0.000	0.000	0.000	entity_match
0.000	0.000	0.000	mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf_plus
0.000	0.000	0.000	b_cubed
0.000	0.000	0.000	b_cubed_plus

ONLY Spanish documents:
Prec	Recall	F1	Metric
0.000	0.000	0.000	strong_mention_match
0.000	0.000	0.000	strong_typed_mention_match
0.000	0.000	0.000	strong_typed_all_match
0.000	0.000	0.000	strong_typed_link_match
0.000	0.000	0.000	strong_typed_nil_match
0.000	0.000	0.000	entity_match
0.000	0.000	0.000	mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf
0.000	0.000	0.000	typed_mention_ceaf_plus
0.000	0.000	0.000	b_cubed
0.000	0.000	0.000	b_cubed_plus