======================================================================= TAC KBP 2015 DIAGNOSTIC TRILINGUAL ENTITY LINKING EVALUATION RESULTS ======================================================================= Team ID: HITS Organization: Heidelberg Institute for Theoretical Studies ************************************************************* Run ID: HITS1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.968 0.424 0.589 strong_mention_match 0.823 0.360 0.501 strong_typed_mention_match 0.679 0.297 0.413 strong_typed_all_match 0.728 0.285 0.410 strong_typed_link_match 0.582 0.332 0.423 strong_typed_nil_match 0.734 0.327 0.452 entity_match 0.788 0.345 0.480 mention_ceaf 0.693 0.303 0.422 typed_mention_ceaf 0.643 0.282 0.392 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.968 0.894 0.930 strong_mention_match 0.823 0.760 0.790 strong_typed_mention_match 0.679 0.627 0.652 strong_typed_all_match 0.728 0.622 0.671 strong_typed_link_match 0.582 0.641 0.610 strong_typed_nil_match 0.734 0.725 0.729 entity_match 0.788 0.728 0.757 mention_ceaf 0.693 0.640 0.665 typed_mention_ceaf 0.643 0.595 0.618 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ************************************************************* Run ID: HITS2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.968 0.423 0.588 strong_mention_match 0.806 0.352 0.490 strong_typed_mention_match 0.653 0.285 0.397 strong_typed_all_match 0.777 0.264 0.394 strong_typed_link_match 0.485 0.345 0.404 strong_typed_nil_match 0.808 0.302 0.440 entity_match 0.735 0.321 0.447 mention_ceaf 0.650 0.284 0.395 typed_mention_ceaf 0.608 0.265 0.369 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.968 0.892 0.928 strong_mention_match 0.806 0.743 0.773 strong_typed_mention_match 0.653 0.602 0.626 strong_typed_all_match 0.777 0.576 0.661 strong_typed_link_match 0.485 0.668 0.562 strong_typed_nil_match 0.808 0.671 0.733 entity_match 0.735 0.677 0.705 mention_ceaf 0.650 0.599 0.624 typed_mention_ceaf 0.608 0.560 0.583 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ************************************************************* Run ID: HITS3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.968 0.424 0.589 strong_mention_match 0.807 0.353 0.491 strong_typed_mention_match 0.652 0.286 0.397 strong_typed_all_match 0.777 0.264 0.394 strong_typed_link_match 0.485 0.347 0.405 strong_typed_nil_match 0.808 0.302 0.440 entity_match 0.733 0.321 0.446 mention_ceaf 0.649 0.284 0.395 typed_mention_ceaf 0.606 0.265 0.369 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.968 0.894 0.930 strong_mention_match 0.807 0.746 0.775 strong_typed_mention_match 0.652 0.603 0.627 strong_typed_all_match 0.777 0.576 0.661 strong_typed_link_match 0.485 0.671 0.563 strong_typed_nil_match 0.808 0.671 0.733 entity_match 0.733 0.677 0.704 mention_ceaf 0.649 0.599 0.623 typed_mention_ceaf 0.606 0.560 0.582 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ************************************************************* Run ID: HITS4 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.968 0.424 0.589 strong_mention_match 0.807 0.353 0.491 strong_typed_mention_match 0.652 0.286 0.397 strong_typed_all_match 0.777 0.264 0.394 strong_typed_link_match 0.485 0.347 0.405 strong_typed_nil_match 0.808 0.302 0.440 entity_match 0.747 0.327 0.455 mention_ceaf 0.659 0.288 0.401 typed_mention_ceaf 0.614 0.269 0.374 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.968 0.894 0.930 strong_mention_match 0.807 0.746 0.775 strong_typed_mention_match 0.652 0.603 0.627 strong_typed_all_match 0.777 0.576 0.661 strong_typed_link_match 0.485 0.671 0.563 strong_typed_nil_match 0.808 0.671 0.733 entity_match 0.747 0.690 0.717 mention_ceaf 0.659 0.609 0.633 typed_mention_ceaf 0.614 0.567 0.590 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ************************************************************* Run ID: HITS5 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.968 0.424 0.589 strong_mention_match 0.823 0.360 0.501 strong_typed_mention_match 0.679 0.297 0.413 strong_typed_all_match 0.728 0.285 0.410 strong_typed_link_match 0.582 0.332 0.423 strong_typed_nil_match 0.734 0.327 0.452 entity_match 0.793 0.347 0.483 mention_ceaf 0.697 0.305 0.425 typed_mention_ceaf 0.651 0.285 0.397 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.968 0.894 0.930 strong_mention_match 0.823 0.760 0.790 strong_typed_mention_match 0.679 0.627 0.652 strong_typed_all_match 0.728 0.622 0.671 strong_typed_link_match 0.582 0.641 0.610 strong_typed_nil_match 0.734 0.725 0.729 entity_match 0.793 0.733 0.762 mention_ceaf 0.697 0.644 0.670 typed_mention_ceaf 0.651 0.602 0.625 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus