======================================================================= TAC KBP 2015 TRILINGUAL ENTITY DISCOVERY AND LINKING EVALUATION RESULTS ======================================================================= Team ID: HITS Organization: Heidelberg Institute for Theoretical Studies ************************************************************* Run ID: HITS1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.830 0.321 0.463 strong_mention_match 0.629 0.244 0.351 strong_typed_mention_match 0.541 0.209 0.302 strong_typed_all_match 0.690 0.251 0.368 strong_typed_link_match 0.204 0.093 0.127 strong_typed_nil_match 0.779 0.279 0.411 entity_match 0.747 0.289 0.417 mention_ceaf 0.572 0.221 0.319 typed_mention_ceaf 0.525 0.203 0.293 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.830 0.678 0.746 strong_mention_match 0.629 0.514 0.566 strong_typed_mention_match 0.541 0.442 0.487 strong_typed_all_match 0.690 0.547 0.610 strong_typed_link_match 0.204 0.179 0.191 strong_typed_nil_match 0.779 0.618 0.690 entity_match 0.747 0.610 0.671 mention_ceaf 0.572 0.467 0.514 typed_mention_ceaf 0.525 0.429 0.472 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ************************************************************* Run ID: HITS2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.824 0.327 0.468 strong_mention_match 0.627 0.249 0.356 strong_typed_mention_match 0.539 0.214 0.306 strong_typed_all_match 0.658 0.258 0.371 strong_typed_link_match 0.213 0.087 0.124 strong_typed_nil_match 0.735 0.288 0.414 entity_match 0.748 0.297 0.425 mention_ceaf 0.576 0.228 0.327 typed_mention_ceaf 0.525 0.208 0.298 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.824 0.689 0.750 strong_mention_match 0.627 0.525 0.571 strong_typed_mention_match 0.539 0.451 0.491 strong_typed_all_match 0.658 0.563 0.607 strong_typed_link_match 0.213 0.168 0.188 strong_typed_nil_match 0.735 0.640 0.684 entity_match 0.748 0.626 0.682 mention_ceaf 0.576 0.482 0.525 typed_mention_ceaf 0.525 0.440 0.479 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ************************************************************* Run ID: HITS3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.806 0.336 0.475 strong_mention_match 0.614 0.256 0.361 strong_typed_mention_match 0.514 0.214 0.303 strong_typed_all_match 0.663 0.257 0.370 strong_typed_link_match 0.188 0.094 0.125 strong_typed_nil_match 0.735 0.288 0.414 entity_match 0.719 0.300 0.423 mention_ceaf 0.554 0.231 0.326 typed_mention_ceaf 0.501 0.209 0.295 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.806 0.710 0.755 strong_mention_match 0.614 0.540 0.574 strong_typed_mention_match 0.514 0.453 0.481 strong_typed_all_match 0.663 0.561 0.608 strong_typed_link_match 0.188 0.181 0.184 strong_typed_nil_match 0.735 0.640 0.684 entity_match 0.719 0.633 0.673 mention_ceaf 0.554 0.488 0.519 typed_mention_ceaf 0.501 0.441 0.469 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ************************************************************* Run ID: HITS4 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.813 0.335 0.474 strong_mention_match 0.617 0.254 0.360 strong_typed_mention_match 0.511 0.210 0.298 strong_typed_all_match 0.695 0.249 0.367 strong_typed_link_match 0.177 0.099 0.127 strong_typed_nil_match 0.779 0.279 0.411 entity_match 0.709 0.292 0.414 mention_ceaf 0.544 0.224 0.317 typed_mention_ceaf 0.495 0.204 0.288 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.813 0.706 0.756 strong_mention_match 0.617 0.536 0.574 strong_typed_mention_match 0.511 0.444 0.475 strong_typed_all_match 0.695 0.544 0.610 strong_typed_link_match 0.177 0.192 0.185 strong_typed_nil_match 0.779 0.618 0.690 entity_match 0.709 0.616 0.659 mention_ceaf 0.544 0.472 0.506 typed_mention_ceaf 0.495 0.430 0.460 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus