======================================================================= TAC KBP 2015 TRILINGUAL ENTITY DISCOVERY AND LINKING EVALUATION RESULTS ======================================================================= Team ID: hltcoe Organization: Human Language Technology Center of Excellence ************************************************************* Run ID: hltcoe1 Did the run access the live Web during the evaluation window: Yes Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.720 0.693 0.706 strong_mention_match 0.660 0.635 0.647 strong_typed_mention_match 0.535 0.515 0.525 strong_typed_all_match 0.631 0.459 0.532 strong_typed_link_match 0.413 0.672 0.512 strong_typed_nil_match 0.747 0.543 0.629 entity_match 0.558 0.536 0.547 mention_ceaf 0.534 0.513 0.523 typed_mention_ceaf 0.464 0.446 0.455 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.748 0.740 0.744 strong_mention_match 0.671 0.663 0.667 strong_typed_mention_match 0.531 0.526 0.528 strong_typed_all_match 0.602 0.520 0.558 strong_typed_link_match 0.413 0.540 0.469 strong_typed_nil_match 0.754 0.583 0.657 entity_match 0.649 0.642 0.645 mention_ceaf 0.610 0.603 0.607 typed_mention_ceaf 0.519 0.514 0.516 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.788 0.687 0.734 strong_mention_match 0.747 0.651 0.696 strong_typed_mention_match 0.612 0.533 0.570 strong_typed_all_match 0.767 0.448 0.565 strong_typed_link_match 0.441 0.840 0.578 strong_typed_nil_match 0.837 0.477 0.608 entity_match 0.710 0.618 0.661 mention_ceaf 0.682 0.594 0.635 typed_mention_ceaf 0.592 0.516 0.551 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.547 0.582 0.564 strong_mention_match 0.498 0.530 0.514 strong_typed_mention_match 0.425 0.452 0.438 strong_typed_all_match 0.485 0.326 0.390 strong_typed_link_match 0.374 0.774 0.505 strong_typed_nil_match 0.656 0.555 0.601 entity_match 0.452 0.481 0.466 mention_ceaf 0.427 0.454 0.440 typed_mention_ceaf 0.378 0.403 0.390 typed_mention_ceaf_plus ************************************************************* Run ID: hltcoe2 Did the run access the live Web during the evaluation window: Yes Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.720 0.693 0.706 strong_mention_match 0.660 0.635 0.647 strong_typed_mention_match 0.536 0.515 0.525 strong_typed_all_match 0.627 0.460 0.531 strong_typed_link_match 0.417 0.672 0.515 strong_typed_nil_match 0.743 0.549 0.631 entity_match 0.559 0.538 0.548 mention_ceaf 0.535 0.515 0.525 typed_mention_ceaf 0.465 0.447 0.456 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.748 0.740 0.744 strong_mention_match 0.671 0.664 0.667 strong_typed_mention_match 0.531 0.525 0.528 strong_typed_all_match 0.597 0.519 0.556 strong_typed_link_match 0.419 0.540 0.472 strong_typed_nil_match 0.748 0.588 0.658 entity_match 0.652 0.645 0.648 mention_ceaf 0.613 0.606 0.609 typed_mention_ceaf 0.519 0.513 0.516 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.788 0.687 0.734 strong_mention_match 0.747 0.651 0.696 strong_typed_mention_match 0.613 0.534 0.571 strong_typed_all_match 0.762 0.449 0.565 strong_typed_link_match 0.445 0.840 0.582 strong_typed_nil_match 0.831 0.483 0.611 entity_match 0.709 0.617 0.660 mention_ceaf 0.680 0.593 0.634 typed_mention_ceaf 0.593 0.517 0.552 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.547 0.582 0.564 strong_mention_match 0.498 0.530 0.514 strong_typed_mention_match 0.427 0.454 0.440 strong_typed_all_match 0.486 0.329 0.392 strong_typed_link_match 0.377 0.774 0.507 strong_typed_nil_match 0.656 0.560 0.604 entity_match 0.452 0.481 0.466 mention_ceaf 0.428 0.455 0.441 typed_mention_ceaf 0.381 0.405 0.392 typed_mention_ceaf_plus ************************************************************* Run ID: hltcoe3 Did the run access the live Web during the evaluation window: Yes Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.720 0.693 0.706 strong_mention_match 0.660 0.635 0.647 strong_typed_mention_match 0.524 0.504 0.513 strong_typed_all_match 0.632 0.444 0.522 strong_typed_link_match 0.396 0.672 0.499 strong_typed_nil_match 0.732 0.518 0.607 entity_match 0.552 0.531 0.541 mention_ceaf 0.527 0.507 0.517 typed_mention_ceaf 0.453 0.435 0.444 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.748 0.740 0.744 strong_mention_match 0.671 0.663 0.667 strong_typed_mention_match 0.521 0.516 0.518 strong_typed_all_match 0.591 0.506 0.545 strong_typed_link_match 0.409 0.541 0.466 strong_typed_nil_match 0.729 0.562 0.635 entity_match 0.657 0.650 0.653 mention_ceaf 0.611 0.605 0.608 typed_mention_ceaf 0.509 0.504 0.506 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.788 0.687 0.734 strong_mention_match 0.747 0.651 0.696 strong_typed_mention_match 0.594 0.518 0.553 strong_typed_all_match 0.793 0.428 0.556 strong_typed_link_match 0.407 0.840 0.548 strong_typed_nil_match 0.818 0.439 0.571 entity_match 0.707 0.616 0.658 mention_ceaf 0.678 0.591 0.632 typed_mention_ceaf 0.574 0.500 0.535 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.547 0.582 0.564 strong_mention_match 0.498 0.530 0.514 strong_typed_mention_match 0.419 0.446 0.432 strong_typed_all_match 0.496 0.318 0.388 strong_typed_link_match 0.360 0.773 0.492 strong_typed_nil_match 0.665 0.540 0.596 entity_match 0.452 0.480 0.466 mention_ceaf 0.427 0.454 0.440 typed_mention_ceaf 0.374 0.397 0.385 typed_mention_ceaf_plus ************************************************************* Run ID: hltcoe4 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.676 0.236 0.349 strong_mention_match 0.633 0.221 0.327 strong_typed_mention_match 0.398 0.139 0.206 strong_typed_all_match 0.606 0.183 0.281 strong_typed_link_match 0.028 0.013 0.018 strong_typed_nil_match 0.668 0.199 0.306 entity_match 0.528 0.184 0.273 mention_ceaf 0.504 0.176 0.261 typed_mention_ceaf 0.397 0.139 0.206 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.700 0.145 0.240 strong_mention_match 0.679 0.141 0.233 strong_typed_mention_match 0.514 0.107 0.176 strong_typed_all_match 0.601 0.146 0.235 strong_typed_link_match 0.068 0.008 0.014 strong_typed_nil_match 0.704 0.179 0.285 entity_match 0.586 0.122 0.201 mention_ceaf 0.582 0.121 0.200 typed_mention_ceaf 0.513 0.106 0.176 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.677 0.443 0.536 strong_mention_match 0.621 0.406 0.491 strong_typed_mention_match 0.367 0.240 0.291 strong_typed_all_match 0.627 0.299 0.405 strong_typed_link_match 0.023 0.030 0.026 strong_typed_nil_match 0.607 0.297 0.398 entity_match 0.587 0.384 0.465 mention_ceaf 0.551 0.361 0.436 typed_mention_ceaf 0.367 0.240 0.290 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.569 0.079 0.138 strong_mention_match 0.558 0.077 0.136 strong_typed_mention_match 0.224 0.031 0.054 strong_typed_all_match 0.430 0.042 0.076 strong_typed_link_match 0.013 0.003 0.005 strong_typed_nil_match 0.837 0.109 0.193 entity_match 0.445 0.061 0.108 mention_ceaf 0.439 0.061 0.107 typed_mention_ceaf 0.224 0.031 0.054 typed_mention_ceaf_plus ************************************************************* Run ID: hltcoe5 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.653 0.229 0.339 strong_mention_match 0.605 0.212 0.314 strong_typed_mention_match 0.395 0.138 0.205 strong_typed_all_match 0.602 0.182 0.280 strong_typed_link_match 0.029 0.014 0.019 strong_typed_nil_match 0.656 0.196 0.301 entity_match 0.510 0.179 0.265 mention_ceaf 0.485 0.170 0.252 typed_mention_ceaf 0.394 0.138 0.204 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.700 0.145 0.240 strong_mention_match 0.679 0.141 0.233 strong_typed_mention_match 0.514 0.107 0.176 strong_typed_all_match 0.601 0.146 0.235 strong_typed_link_match 0.068 0.008 0.014 strong_typed_nil_match 0.704 0.179 0.285 entity_match 0.586 0.122 0.201 mention_ceaf 0.582 0.121 0.200 typed_mention_ceaf 0.513 0.106 0.176 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.642 0.423 0.510 strong_mention_match 0.579 0.381 0.460 strong_typed_mention_match 0.362 0.239 0.288 strong_typed_all_match 0.620 0.296 0.401 strong_typed_link_match 0.025 0.033 0.028 strong_typed_nil_match 0.583 0.286 0.384 entity_match 0.560 0.369 0.445 mention_ceaf 0.520 0.343 0.413 typed_mention_ceaf 0.362 0.239 0.287 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.569 0.079 0.138 strong_mention_match 0.558 0.077 0.136 strong_typed_mention_match 0.224 0.031 0.054 strong_typed_all_match 0.430 0.042 0.076 strong_typed_link_match 0.013 0.003 0.005 strong_typed_nil_match 0.837 0.109 0.193 entity_match 0.445 0.061 0.108 mention_ceaf 0.439 0.061 0.107 typed_mention_ceaf 0.224 0.031 0.054 typed_mention_ceaf_plus