=======================================================================
TAC KBP 2015 TRILINGUAL ENTITY DISCOVERY AND LINKING EVALUATION RESULTS
=======================================================================


Team ID:  CMU_Edvisees
Organization:  CMU


*************************************************************

Run ID:  CMU_Edvisees1
Did the run access the live Web during the evaluation window:  No
Did the run use the Wikipedia links in the Reference KB: No
Did the run use the relation/slot information encoded in the Reference KB: Yes
Did the run use the text descriptions in the nodes of the Reference KB: No

ALL English, Chinese, and Spanish documents:
Prec	Recall	F1	Metric
0.562	0.635	0.597	strong_mention_match
0.502	0.567	0.532	strong_typed_mention_match
0.379	0.428	0.402	strong_typed_all_match
0.330	0.360	0.344	strong_typed_link_match
0.501	0.618	0.554	strong_typed_nil_match
0.270	0.415	0.327	entity_match
0.435	0.491	0.461	mention_ceaf
0.405	0.458	0.430	typed_mention_ceaf
0.361	0.408	0.383	typed_mention_ceaf_plus

ONLY English documents:
Prec	Recall	F1	Metric
0.534	0.589	0.560	strong_mention_match
0.468	0.517	0.491	strong_typed_mention_match
0.311	0.343	0.326	strong_typed_all_match
0.255	0.280	0.267	strong_typed_link_match
0.448	0.500	0.473	strong_typed_nil_match
0.241	0.402	0.302	entity_match
0.420	0.463	0.441	mention_ceaf
0.382	0.422	0.401	typed_mention_ceaf
0.301	0.332	0.316	typed_mention_ceaf_plus

ONLY Chinese documents:
Prec	Recall	F1	Metric
0.549	0.675	0.606	strong_mention_match
0.500	0.614	0.551	strong_typed_mention_match
0.429	0.527	0.473	strong_typed_all_match
0.387	0.473	0.426	strong_typed_link_match
0.575	0.720	0.639	strong_typed_nil_match
0.257	0.461	0.330	entity_match
0.489	0.601	0.539	mention_ceaf
0.456	0.560	0.503	typed_mention_ceaf
0.418	0.513	0.460	typed_mention_ceaf_plus

ONLY Spanish documents:
Prec	Recall	F1	Metric
0.675	0.681	0.678	strong_mention_match
0.602	0.608	0.605	strong_typed_mention_match
0.456	0.460	0.458	strong_typed_all_match
0.412	0.335	0.369	strong_typed_link_match
0.516	0.779	0.621	strong_typed_nil_match
0.396	0.379	0.388	entity_match
0.541	0.545	0.543	mention_ceaf
0.497	0.501	0.499	typed_mention_ceaf
0.405	0.409	0.407	typed_mention_ceaf_plus


*************************************************************

Run ID:  CMU_Edvisees2
Did the run access the live Web during the evaluation window:  No
Did the run use the Wikipedia links in the Reference KB: No
Did the run use the relation/slot information encoded in the Reference KB: Yes
Did the run use the text descriptions in the nodes of the Reference KB: No

ALL English, Chinese, and Spanish documents:
Prec	Recall	F1	Metric
0.542	0.638	0.586	strong_mention_match
0.481	0.566	0.520	strong_typed_mention_match
0.368	0.433	0.398	strong_typed_all_match
0.313	0.368	0.338	strong_typed_link_match
0.523	0.616	0.565	strong_typed_nil_match
0.255	0.431	0.320	entity_match
0.423	0.498	0.457	mention_ceaf
0.392	0.462	0.424	typed_mention_ceaf
0.351	0.413	0.380	typed_mention_ceaf_plus

ONLY English documents:
Prec	Recall	F1	Metric
0.503	0.593	0.544	strong_mention_match
0.436	0.514	0.472	strong_typed_mention_match
0.297	0.351	0.322	strong_typed_all_match
0.237	0.292	0.262	strong_typed_link_match
0.477	0.496	0.487	strong_typed_nil_match
0.222	0.426	0.292	entity_match
0.400	0.471	0.433	mention_ceaf
0.361	0.426	0.391	typed_mention_ceaf
0.289	0.340	0.312	typed_mention_ceaf_plus

ONLY Chinese documents:
Prec	Recall	F1	Metric
0.549	0.675	0.606	strong_mention_match
0.500	0.614	0.551	strong_typed_mention_match
0.429	0.527	0.473	strong_typed_all_match
0.387	0.473	0.426	strong_typed_link_match
0.575	0.720	0.639	strong_typed_nil_match
0.257	0.461	0.330	entity_match
0.489	0.601	0.539	mention_ceaf
0.456	0.560	0.503	typed_mention_ceaf
0.418	0.513	0.460	typed_mention_ceaf_plus

ONLY Spanish documents:
Prec	Recall	F1	Metric
0.640	0.688	0.663	strong_mention_match
0.567	0.609	0.587	strong_typed_mention_match
0.437	0.469	0.452	strong_typed_all_match
0.373	0.348	0.360	strong_typed_link_match
0.543	0.778	0.639	strong_typed_nil_match
0.354	0.400	0.376	entity_match
0.511	0.549	0.529	mention_ceaf
0.468	0.503	0.485	typed_mention_ceaf
0.390	0.419	0.404	typed_mention_ceaf_plus