================================================================================ TAC KBP 2016 ENTITY DISCOVERY AND LINKING EVALUATION RESULTS (SECOND EDL WINDOW) ================================================================================ Team ID: RPI_BLENDER Organization: Rensselaer Polytechnic Institute ************************************************************* Run ID: RPI_BLENDER1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No Did the run use any of the distributed EDL1 runs (from the first evaluation window): No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.812 0.541 0.649 strong_typed_mention_match 0.743 0.495 0.594 strong_typed_all_match 0.735 0.473 0.575 strong_typed_link_match 0.768 0.573 0.656 strong_typed_nil_match 0.725 0.611 0.663 entity_match 0.798 0.531 0.638 mention_ceaf 0.759 0.506 0.607 typed_mention_ceaf 0.722 0.481 0.577 typed_mention_ceaf_plus 0.799 0.399 0.533 b_cubed 0.718 0.371 0.489 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.934 0.575 0.711 strong_mention_match 0.883 0.543 0.673 strong_typed_mention_match 0.815 0.501 0.621 strong_typed_all_match 0.789 0.492 0.606 strong_typed_link_match 0.912 0.533 0.672 strong_typed_nil_match 0.760 0.650 0.701 entity_match 0.874 0.537 0.665 mention_ceaf 0.831 0.511 0.633 typed_mention_ceaf 0.793 0.488 0.604 typed_mention_ceaf_plus 0.882 0.407 0.557 b_cubed 0.807 0.382 0.519 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.814 0.567 0.668 strong_mention_match 0.775 0.540 0.637 strong_typed_mention_match 0.708 0.493 0.581 strong_typed_all_match 0.720 0.476 0.573 strong_typed_link_match 0.671 0.561 0.611 strong_typed_nil_match 0.742 0.580 0.651 entity_match 0.788 0.549 0.647 mention_ceaf 0.757 0.527 0.621 typed_mention_ceaf 0.700 0.487 0.575 typed_mention_ceaf_plus 0.759 0.429 0.548 b_cubed 0.688 0.393 0.500 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.854 0.594 0.701 strong_mention_match 0.775 0.539 0.635 strong_typed_mention_match 0.703 0.489 0.577 strong_typed_all_match 0.683 0.440 0.535 strong_typed_link_match 0.748 0.628 0.683 strong_typed_nil_match 0.669 0.596 0.631 entity_match 0.760 0.528 0.623 mention_ceaf 0.708 0.493 0.581 typed_mention_ceaf 0.667 0.464 0.547 typed_mention_ceaf_plus 0.769 0.414 0.539 b_cubed 0.663 0.375 0.479 b_cubed_plus ************************************************************* Run ID: RPI_BLENDER2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No Did the run use any of the distributed EDL1 runs (from the first evaluation window): No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.812 0.541 0.649 strong_typed_mention_match 0.743 0.495 0.594 strong_typed_all_match 0.735 0.473 0.575 strong_typed_link_match 0.769 0.572 0.656 strong_typed_nil_match 0.725 0.611 0.664 entity_match 0.798 0.531 0.638 mention_ceaf 0.759 0.505 0.607 typed_mention_ceaf 0.722 0.481 0.577 typed_mention_ceaf_plus 0.799 0.399 0.532 b_cubed 0.719 0.371 0.489 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.935 0.574 0.711 strong_mention_match 0.884 0.543 0.673 strong_typed_mention_match 0.816 0.501 0.621 strong_typed_all_match 0.789 0.493 0.607 strong_typed_link_match 0.919 0.530 0.672 strong_typed_nil_match 0.761 0.651 0.702 entity_match 0.876 0.537 0.666 mention_ceaf 0.833 0.511 0.633 typed_mention_ceaf 0.794 0.487 0.604 typed_mention_ceaf_plus 0.884 0.406 0.557 b_cubed 0.809 0.382 0.519 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.814 0.567 0.668 strong_mention_match 0.775 0.540 0.637 strong_typed_mention_match 0.708 0.493 0.581 strong_typed_all_match 0.720 0.476 0.573 strong_typed_link_match 0.671 0.561 0.611 strong_typed_nil_match 0.742 0.580 0.651 entity_match 0.788 0.549 0.647 mention_ceaf 0.757 0.527 0.621 typed_mention_ceaf 0.700 0.487 0.575 typed_mention_ceaf_plus 0.759 0.429 0.548 b_cubed 0.688 0.393 0.500 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.854 0.594 0.701 strong_mention_match 0.775 0.539 0.635 strong_typed_mention_match 0.703 0.489 0.577 strong_typed_all_match 0.683 0.440 0.535 strong_typed_link_match 0.748 0.628 0.683 strong_typed_nil_match 0.669 0.596 0.631 entity_match 0.760 0.528 0.623 mention_ceaf 0.708 0.493 0.581 typed_mention_ceaf 0.667 0.464 0.547 typed_mention_ceaf_plus 0.769 0.414 0.539 b_cubed 0.663 0.375 0.479 b_cubed_plus ************************************************************* Run ID: RPI_BLENDER3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No Did the run use any of the distributed EDL1 runs (from the first evaluation window): No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.832 0.484 0.612 strong_typed_mention_match 0.764 0.445 0.563 strong_typed_all_match 0.754 0.417 0.537 strong_typed_link_match 0.792 0.545 0.646 strong_typed_nil_match 0.710 0.563 0.628 entity_match 0.807 0.470 0.594 mention_ceaf 0.773 0.450 0.569 typed_mention_ceaf 0.740 0.431 0.545 typed_mention_ceaf_plus 0.808 0.332 0.471 b_cubed 0.733 0.311 0.437 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.935 0.574 0.711 strong_mention_match 0.884 0.543 0.673 strong_typed_mention_match 0.816 0.501 0.621 strong_typed_all_match 0.789 0.493 0.607 strong_typed_link_match 0.919 0.530 0.672 strong_typed_nil_match 0.761 0.651 0.702 entity_match 0.876 0.537 0.666 mention_ceaf 0.833 0.511 0.633 typed_mention_ceaf 0.794 0.487 0.604 typed_mention_ceaf_plus 0.884 0.406 0.557 b_cubed 0.809 0.382 0.519 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.855 0.454 0.593 strong_mention_match 0.820 0.435 0.568 strong_typed_mention_match 0.761 0.404 0.527 strong_typed_all_match 0.747 0.383 0.507 strong_typed_link_match 0.808 0.487 0.608 strong_typed_nil_match 0.683 0.499 0.577 entity_match 0.822 0.436 0.570 mention_ceaf 0.793 0.420 0.549 typed_mention_ceaf 0.751 0.398 0.520 typed_mention_ceaf_plus 0.787 0.313 0.448 b_cubed 0.725 0.291 0.415 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.824 0.500 0.623 strong_mention_match 0.774 0.470 0.585 strong_typed_mention_match 0.699 0.424 0.528 strong_typed_all_match 0.705 0.356 0.473 strong_typed_link_match 0.691 0.617 0.652 strong_typed_nil_match 0.668 0.523 0.587 entity_match 0.725 0.440 0.547 mention_ceaf 0.694 0.421 0.524 typed_mention_ceaf 0.656 0.398 0.496 typed_mention_ceaf_plus 0.750 0.326 0.455 b_cubed 0.656 0.300 0.411 b_cubed_plus