======================================================================== TAC KBP 2017 TRI-LINGUAL ENTITY DISCOVERY AND LINKING EVALUATION RESULTS ======================================================================== Team ID: RPI_BLENDER Organization: Rensselaer Polytechnic Institute ************************************************************* Run ID: RPI_BLENDER1 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.894 0.602 0.719 strong_mention_match 0.858 0.578 0.691 strong_typed_mention_match 0.753 0.508 0.607 strong_typed_all_match 0.769 0.524 0.624 strong_typed_link_match 0.714 0.468 0.565 strong_typed_nil_match 0.745 0.652 0.695 entity_match 0.747 0.504 0.602 mention_ceaf 0.726 0.489 0.585 typed_mention_ceaf 0.674 0.454 0.543 typed_mention_ceaf_plus 0.728 0.434 0.543 b_cubed 0.632 0.392 0.484 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.890 0.634 0.740 strong_mention_match 0.852 0.607 0.709 strong_typed_mention_match 0.732 0.521 0.609 strong_typed_all_match 0.694 0.539 0.607 strong_typed_link_match 0.829 0.486 0.613 strong_typed_nil_match 0.692 0.682 0.687 entity_match 0.787 0.561 0.655 mention_ceaf 0.762 0.542 0.634 typed_mention_ceaf 0.686 0.489 0.571 typed_mention_ceaf_plus 0.803 0.482 0.602 b_cubed 0.674 0.424 0.520 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.876 0.581 0.699 strong_mention_match 0.853 0.566 0.680 strong_typed_mention_match 0.773 0.513 0.617 strong_typed_all_match 0.830 0.553 0.664 strong_typed_link_match 0.601 0.393 0.475 strong_typed_nil_match 0.825 0.681 0.746 entity_match 0.834 0.553 0.665 mention_ceaf 0.815 0.541 0.650 typed_mention_ceaf 0.763 0.506 0.609 typed_mention_ceaf_plus 0.823 0.432 0.566 b_cubed 0.748 0.394 0.516 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.922 0.602 0.729 strong_mention_match 0.872 0.569 0.689 strong_typed_mention_match 0.747 0.488 0.590 strong_typed_all_match 0.755 0.466 0.576 strong_typed_link_match 0.733 0.536 0.619 strong_typed_nil_match 0.712 0.597 0.649 entity_match 0.778 0.508 0.614 mention_ceaf 0.747 0.487 0.590 typed_mention_ceaf 0.680 0.444 0.537 typed_mention_ceaf_plus 0.814 0.434 0.566 b_cubed 0.675 0.388 0.493 b_cubed_plus ************************************************************* Run ID: RPI_BLENDER2 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.877 0.606 0.717 strong_mention_match 0.839 0.579 0.685 strong_typed_mention_match 0.730 0.504 0.597 strong_typed_all_match 0.740 0.519 0.610 strong_typed_link_match 0.705 0.469 0.563 strong_typed_nil_match 0.737 0.645 0.688 entity_match 0.733 0.507 0.599 mention_ceaf 0.710 0.491 0.580 typed_mention_ceaf 0.654 0.452 0.534 typed_mention_ceaf_plus 0.702 0.437 0.538 b_cubed 0.606 0.392 0.476 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.845 0.638 0.727 strong_mention_match 0.803 0.607 0.691 strong_typed_mention_match 0.679 0.512 0.584 strong_typed_all_match 0.628 0.526 0.573 strong_typed_link_match 0.819 0.485 0.609 strong_typed_nil_match 0.677 0.667 0.672 entity_match 0.746 0.563 0.642 mention_ceaf 0.718 0.542 0.618 typed_mention_ceaf 0.636 0.480 0.547 typed_mention_ceaf_plus 0.729 0.487 0.584 b_cubed 0.605 0.421 0.497 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.871 0.587 0.702 strong_mention_match 0.843 0.568 0.679 strong_typed_mention_match 0.758 0.511 0.610 strong_typed_all_match 0.815 0.550 0.657 strong_typed_link_match 0.586 0.394 0.471 strong_typed_nil_match 0.818 0.675 0.740 entity_match 0.825 0.556 0.664 mention_ceaf 0.803 0.541 0.646 typed_mention_ceaf 0.748 0.504 0.602 typed_mention_ceaf_plus 0.809 0.436 0.567 b_cubed 0.730 0.395 0.513 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.922 0.602 0.729 strong_mention_match 0.872 0.569 0.689 strong_typed_mention_match 0.747 0.488 0.590 strong_typed_all_match 0.754 0.465 0.575 strong_typed_link_match 0.734 0.537 0.620 strong_typed_nil_match 0.710 0.594 0.647 entity_match 0.779 0.508 0.615 mention_ceaf 0.747 0.488 0.590 typed_mention_ceaf 0.679 0.443 0.536 typed_mention_ceaf_plus 0.815 0.434 0.566 b_cubed 0.675 0.388 0.493 b_cubed_plus ************************************************************* Run ID: RPI_BLENDER3 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.894 0.602 0.719 strong_mention_match 0.858 0.578 0.691 strong_typed_mention_match 0.748 0.504 0.602 strong_typed_all_match 0.762 0.519 0.617 strong_typed_link_match 0.712 0.468 0.565 strong_typed_nil_match 0.737 0.644 0.688 entity_match 0.746 0.502 0.600 mention_ceaf 0.725 0.488 0.583 typed_mention_ceaf 0.667 0.450 0.537 typed_mention_ceaf_plus 0.727 0.433 0.543 b_cubed 0.626 0.389 0.480 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.890 0.634 0.740 strong_mention_match 0.852 0.607 0.709 strong_typed_mention_match 0.720 0.513 0.599 strong_typed_all_match 0.678 0.526 0.593 strong_typed_link_match 0.828 0.486 0.613 strong_typed_nil_match 0.679 0.667 0.673 entity_match 0.787 0.560 0.655 mention_ceaf 0.761 0.542 0.633 typed_mention_ceaf 0.675 0.480 0.561 typed_mention_ceaf_plus 0.803 0.482 0.602 b_cubed 0.663 0.418 0.513 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.876 0.581 0.699 strong_mention_match 0.853 0.566 0.680 strong_typed_mention_match 0.768 0.510 0.613 strong_typed_all_match 0.825 0.549 0.659 strong_typed_link_match 0.596 0.392 0.473 strong_typed_nil_match 0.818 0.675 0.740 entity_match 0.834 0.553 0.665 mention_ceaf 0.815 0.540 0.650 typed_mention_ceaf 0.758 0.503 0.605 typed_mention_ceaf_plus 0.822 0.432 0.566 b_cubed 0.742 0.392 0.513 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.922 0.602 0.729 strong_mention_match 0.872 0.569 0.689 strong_typed_mention_match 0.746 0.487 0.589 strong_typed_all_match 0.753 0.465 0.575 strong_typed_link_match 0.733 0.536 0.620 strong_typed_nil_match 0.708 0.593 0.645 entity_match 0.778 0.508 0.614 mention_ceaf 0.746 0.487 0.589 typed_mention_ceaf 0.678 0.443 0.536 typed_mention_ceaf_plus 0.814 0.434 0.566 b_cubed 0.674 0.387 0.492 b_cubed_plus ************************************************************* Run ID: RPI_BLENDER4 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.894 0.602 0.719 strong_mention_match 0.857 0.578 0.690 strong_typed_mention_match 0.737 0.497 0.594 strong_typed_all_match 0.748 0.509 0.605 strong_typed_link_match 0.711 0.469 0.565 strong_typed_nil_match 0.727 0.639 0.680 entity_match 0.745 0.502 0.599 mention_ceaf 0.723 0.487 0.582 typed_mention_ceaf 0.659 0.444 0.531 typed_mention_ceaf_plus 0.730 0.431 0.542 b_cubed 0.620 0.383 0.474 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.890 0.634 0.740 strong_mention_match 0.850 0.605 0.707 strong_typed_mention_match 0.708 0.504 0.588 strong_typed_all_match 0.660 0.513 0.577 strong_typed_link_match 0.830 0.485 0.613 strong_typed_nil_match 0.667 0.661 0.664 entity_match 0.781 0.556 0.649 mention_ceaf 0.754 0.537 0.627 typed_mention_ceaf 0.662 0.472 0.551 typed_mention_ceaf_plus 0.801 0.478 0.599 b_cubed 0.653 0.410 0.504 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.876 0.581 0.699 strong_mention_match 0.853 0.565 0.680 strong_typed_mention_match 0.763 0.506 0.609 strong_typed_all_match 0.819 0.544 0.654 strong_typed_link_match 0.596 0.393 0.473 strong_typed_nil_match 0.810 0.669 0.733 entity_match 0.833 0.552 0.664 mention_ceaf 0.813 0.539 0.649 typed_mention_ceaf 0.753 0.499 0.601 typed_mention_ceaf_plus 0.822 0.431 0.565 b_cubed 0.738 0.388 0.509 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.922 0.602 0.729 strong_mention_match 0.871 0.569 0.688 strong_typed_mention_match 0.731 0.477 0.577 strong_typed_all_match 0.731 0.450 0.557 strong_typed_link_match 0.731 0.538 0.620 strong_typed_nil_match 0.699 0.587 0.638 entity_match 0.776 0.506 0.613 mention_ceaf 0.743 0.485 0.587 typed_mention_ceaf 0.663 0.432 0.523 typed_mention_ceaf_plus 0.814 0.431 0.564 b_cubed 0.659 0.379 0.481 b_cubed_plus ************************************************************* Run ID: RPI_BLENDER5 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.894 0.602 0.719 strong_mention_match 0.858 0.578 0.691 strong_typed_mention_match 0.748 0.504 0.602 strong_typed_all_match 0.762 0.519 0.617 strong_typed_link_match 0.712 0.468 0.565 strong_typed_nil_match 0.737 0.645 0.688 entity_match 0.748 0.504 0.602 mention_ceaf 0.726 0.489 0.585 typed_mention_ceaf 0.668 0.450 0.538 typed_mention_ceaf_plus 0.729 0.433 0.543 b_cubed 0.628 0.389 0.481 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.890 0.634 0.740 strong_mention_match 0.852 0.607 0.709 strong_typed_mention_match 0.720 0.513 0.599 strong_typed_all_match 0.678 0.526 0.592 strong_typed_link_match 0.829 0.486 0.613 strong_typed_nil_match 0.679 0.669 0.674 entity_match 0.787 0.560 0.654 mention_ceaf 0.761 0.542 0.633 typed_mention_ceaf 0.675 0.480 0.561 typed_mention_ceaf_plus 0.802 0.482 0.602 b_cubed 0.663 0.418 0.513 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.876 0.581 0.699 strong_mention_match 0.853 0.566 0.680 strong_typed_mention_match 0.769 0.510 0.613 strong_typed_all_match 0.826 0.549 0.659 strong_typed_link_match 0.597 0.392 0.473 strong_typed_nil_match 0.818 0.675 0.740 entity_match 0.834 0.553 0.665 mention_ceaf 0.815 0.540 0.650 typed_mention_ceaf 0.758 0.503 0.605 typed_mention_ceaf_plus 0.822 0.432 0.566 b_cubed 0.742 0.392 0.513 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.922 0.602 0.729 strong_mention_match 0.872 0.569 0.689 strong_typed_mention_match 0.746 0.487 0.589 strong_typed_all_match 0.753 0.465 0.575 strong_typed_link_match 0.734 0.536 0.620 strong_typed_nil_match 0.708 0.594 0.646 entity_match 0.779 0.508 0.615 mention_ceaf 0.747 0.487 0.590 typed_mention_ceaf 0.679 0.443 0.536 typed_mention_ceaf_plus 0.814 0.434 0.566 b_cubed 0.674 0.387 0.492 b_cubed_plus