=======================================================================
TAC KBP 2015 DIAGNOSTIC TRILINGUAL ENTITY LINKING EVALUATION RESULTS
=======================================================================


Team ID:  CMU_Edvisees
Organization:  CMU


*************************************************************

Run ID:  CMU_Edvisees1
Did the run access the live Web during the evaluation window:  No
Did the run use the Wikipedia links in the Reference KB: No
Did the run use the relation/slot information encoded in the Reference KB: Yes
Did the run use the text descriptions in the nodes of the Reference KB: No

ALL English, Chinese, and Spanish documents:
Prec	Recall	F1	Metric
0.984	0.951	0.967	strong_mention_match
0.684	0.661	0.672	strong_typed_mention_match
0.475	0.459	0.467	strong_typed_all_match
0.700	0.368	0.483	strong_typed_link_match
0.323	0.715	0.445	strong_typed_nil_match
0.682	0.401	0.505	entity_match
0.628	0.607	0.617	mention_ceaf
0.509	0.492	0.500	typed_mention_ceaf
0.447	0.432	0.439	typed_mention_ceaf_plus

ONLY English documents:
Prec	Recall	F1	Metric
0.965	0.896	0.930	strong_mention_match
0.659	0.612	0.634	strong_typed_mention_match
0.410	0.381	0.395	strong_typed_all_match
0.598	0.293	0.393	strong_typed_link_match
0.296	0.601	0.397	strong_typed_nil_match
0.602	0.385	0.470	entity_match
0.662	0.615	0.638	mention_ceaf
0.499	0.463	0.480	typed_mention_ceaf
0.389	0.361	0.374	typed_mention_ceaf_plus

ONLY Chinese documents:
Prec	Recall	F1	Metric
1.000	1.000	1.000	strong_mention_match
0.706	0.706	0.706	strong_typed_mention_match
0.549	0.549	0.549	strong_typed_all_match
0.793	0.473	0.593	strong_typed_link_match
0.336	0.824	0.477	strong_typed_nil_match
0.775	0.437	0.559	entity_match
0.768	0.768	0.768	mention_ceaf
0.605	0.605	0.605	typed_mention_ceaf
0.532	0.532	0.532	typed_mention_ceaf_plus

ONLY Spanish documents:
Prec	Recall	F1	Metric
1.000	1.000	1.000	strong_mention_match
0.704	0.704	0.704	strong_typed_mention_match
0.491	0.491	0.491	strong_typed_all_match
0.731	0.348	0.472	strong_typed_link_match
0.365	0.856	0.512	strong_typed_nil_match
0.737	0.383	0.504	entity_match
0.697	0.697	0.697	mention_ceaf
0.541	0.541	0.541	typed_mention_ceaf
0.431	0.431	0.431	typed_mention_ceaf_plus


*************************************************************

Run ID:  CMU_Edvisees2
Did the run access the live Web during the evaluation window:  No
Did the run use the Wikipedia links in the Reference KB: No
Did the run use the relation/slot information encoded in the Reference KB: Yes
Did the run use the text descriptions in the nodes of the Reference KB: No

ALL English, Chinese, and Spanish documents:
Prec	Recall	F1	Metric
0.984	0.950	0.967	strong_mention_match
0.689	0.665	0.677	strong_typed_mention_match
0.471	0.455	0.463	strong_typed_all_match
0.712	0.361	0.479	strong_typed_link_match
0.318	0.721	0.441	strong_typed_nil_match
0.696	0.386	0.496	entity_match
0.626	0.604	0.615	mention_ceaf
0.507	0.489	0.498	typed_mention_ceaf
0.442	0.427	0.434	typed_mention_ceaf_plus

ONLY English documents:
Prec	Recall	F1	Metric
0.965	0.896	0.929	strong_mention_match
0.668	0.620	0.643	strong_typed_mention_match
0.405	0.375	0.389	strong_typed_all_match
0.612	0.281	0.385	strong_typed_link_match
0.291	0.612	0.394	strong_typed_nil_match
0.614	0.362	0.456	entity_match
0.659	0.611	0.634	mention_ceaf
0.499	0.463	0.481	typed_mention_ceaf
0.382	0.354	0.367	typed_mention_ceaf_plus

ONLY Chinese documents:
Prec	Recall	F1	Metric
1.000	1.000	1.000	strong_mention_match
0.706	0.706	0.706	strong_typed_mention_match
0.549	0.549	0.549	strong_typed_all_match
0.793	0.473	0.593	strong_typed_link_match
0.336	0.824	0.477	strong_typed_nil_match
0.775	0.437	0.559	entity_match
0.768	0.768	0.768	mention_ceaf
0.605	0.605	0.605	typed_mention_ceaf
0.532	0.532	0.532	typed_mention_ceaf_plus

ONLY Spanish documents:
Prec	Recall	F1	Metric
1.000	1.000	1.000	strong_mention_match
0.705	0.705	0.705	strong_typed_mention_match
0.482	0.482	0.482	strong_typed_all_match
0.758	0.335	0.464	strong_typed_link_match
0.353	0.859	0.501	strong_typed_nil_match
0.764	0.362	0.492	entity_match
0.699	0.699	0.699	mention_ceaf
0.542	0.542	0.542	typed_mention_ceaf
0.422	0.422	0.422	typed_mention_ceaf_plus