=============================================================================== TAC KBP 2016 ENTITY DISCOVERY AND LINKING EVALUATION RESULTS (FIRST EDL WINDOW) =============================================================================== Team ID: HITS Organization: Heidelberg Institute for Theoretical Studies gGmbH ************************************************************* Run ID: HITS1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.699 0.193 0.302 strong_typed_mention_match 0.622 0.172 0.269 strong_typed_all_match 0.767 0.163 0.269 strong_typed_link_match 0.404 0.200 0.268 strong_typed_nil_match 0.826 0.218 0.345 entity_match 0.723 0.200 0.313 mention_ceaf 0.651 0.180 0.281 typed_mention_ceaf 0.618 0.170 0.267 typed_mention_ceaf_plus 0.855 0.107 0.190 b_cubed 0.654 0.097 0.168 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.889 0.665 0.761 strong_mention_match 0.699 0.523 0.598 strong_typed_mention_match 0.622 0.466 0.533 strong_typed_all_match 0.767 0.441 0.560 strong_typed_link_match 0.404 0.552 0.467 strong_typed_nil_match 0.826 0.584 0.684 entity_match 0.723 0.541 0.619 mention_ceaf 0.651 0.487 0.557 typed_mention_ceaf 0.618 0.463 0.529 typed_mention_ceaf_plus 0.855 0.423 0.566 b_cubed 0.654 0.386 0.486 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ************************************************************* Run ID: HITS2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.711 0.193 0.303 strong_typed_mention_match 0.634 0.172 0.270 strong_typed_all_match 0.792 0.163 0.271 strong_typed_link_match 0.404 0.200 0.268 strong_typed_nil_match 0.855 0.216 0.345 entity_match 0.731 0.198 0.312 mention_ceaf 0.662 0.179 0.282 typed_mention_ceaf 0.630 0.170 0.268 typed_mention_ceaf_plus 0.865 0.106 0.189 b_cubed 0.664 0.096 0.168 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.898 0.660 0.761 strong_mention_match 0.711 0.522 0.602 strong_typed_mention_match 0.634 0.466 0.537 strong_typed_all_match 0.792 0.441 0.567 strong_typed_link_match 0.404 0.552 0.467 strong_typed_nil_match 0.855 0.579 0.690 entity_match 0.731 0.537 0.619 mention_ceaf 0.662 0.486 0.561 typed_mention_ceaf 0.630 0.463 0.533 typed_mention_ceaf_plus 0.865 0.419 0.565 b_cubed 0.664 0.385 0.488 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ************************************************************* Run ID: HITS3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.711 0.193 0.303 strong_typed_mention_match 0.634 0.172 0.270 strong_typed_all_match 0.792 0.163 0.271 strong_typed_link_match 0.404 0.200 0.268 strong_typed_nil_match 0.855 0.216 0.345 entity_match 0.739 0.200 0.315 mention_ceaf 0.668 0.181 0.285 typed_mention_ceaf 0.630 0.170 0.268 typed_mention_ceaf_plus 0.860 0.107 0.190 b_cubed 0.662 0.097 0.169 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.898 0.660 0.761 strong_mention_match 0.711 0.522 0.602 strong_typed_mention_match 0.634 0.466 0.537 strong_typed_all_match 0.792 0.441 0.567 strong_typed_link_match 0.404 0.552 0.467 strong_typed_nil_match 0.855 0.579 0.690 entity_match 0.739 0.543 0.626 mention_ceaf 0.668 0.491 0.566 typed_mention_ceaf 0.630 0.463 0.533 typed_mention_ceaf_plus 0.860 0.424 0.568 b_cubed 0.662 0.386 0.488 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus