======================================================================= TAC KBP 2015 DIAGNOSTIC TRILINGUAL ENTITY LINKING EVALUATION RESULTS ======================================================================= Team ID: RPI_BLENDER Organization: Rensselear Polytechnic Institute ************************************************************* Run ID: RPI_BLENDER1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.986 1.000 0.993 strong_mention_match 0.869 0.881 0.875 strong_typed_mention_match 0.711 0.722 0.716 strong_typed_all_match 0.696 0.724 0.710 strong_typed_link_match 0.760 0.713 0.736 strong_typed_nil_match 0.655 0.764 0.706 entity_match 0.811 0.823 0.817 mention_ceaf 0.748 0.759 0.753 typed_mention_ceaf 0.682 0.692 0.687 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.971 1.000 0.985 strong_mention_match 0.856 0.881 0.868 strong_typed_mention_match 0.661 0.681 0.671 strong_typed_all_match 0.630 0.694 0.661 strong_typed_link_match 0.764 0.648 0.701 strong_typed_nil_match 0.627 0.799 0.703 entity_match 0.790 0.814 0.802 mention_ceaf 0.713 0.735 0.724 typed_mention_ceaf 0.631 0.650 0.640 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 1.000 1.000 1.000 strong_mention_match 0.891 0.891 0.891 strong_typed_mention_match 0.781 0.781 0.781 strong_typed_all_match 0.794 0.776 0.785 strong_typed_link_match 0.738 0.797 0.767 strong_typed_nil_match 0.722 0.726 0.724 entity_match 0.899 0.899 0.899 mention_ceaf 0.831 0.831 0.831 typed_mention_ceaf 0.764 0.764 0.764 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 1.000 1.000 1.000 strong_mention_match 0.861 0.861 0.861 strong_typed_mention_match 0.715 0.715 0.715 strong_typed_all_match 0.688 0.696 0.692 strong_typed_link_match 0.785 0.762 0.773 strong_typed_nil_match 0.639 0.749 0.690 entity_match 0.806 0.806 0.806 mention_ceaf 0.745 0.745 0.745 typed_mention_ceaf 0.666 0.666 0.666 typed_mention_ceaf_plus ************************************************************* Run ID: RPI_BLENDER2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.986 1.000 0.993 strong_mention_match 0.868 0.880 0.874 strong_typed_mention_match 0.673 0.683 0.678 strong_typed_all_match 0.657 0.686 0.671 strong_typed_link_match 0.724 0.674 0.698 strong_typed_nil_match 0.642 0.751 0.692 entity_match 0.797 0.809 0.803 mention_ceaf 0.734 0.745 0.740 typed_mention_ceaf 0.646 0.655 0.651 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.971 1.000 0.985 strong_mention_match 0.855 0.880 0.867 strong_typed_mention_match 0.629 0.648 0.638 strong_typed_all_match 0.598 0.674 0.634 strong_typed_link_match 0.740 0.581 0.651 strong_typed_nil_match 0.621 0.793 0.697 entity_match 0.789 0.812 0.800 mention_ceaf 0.712 0.733 0.722 typed_mention_ceaf 0.603 0.621 0.612 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 1.000 1.000 1.000 strong_mention_match 0.891 0.891 0.891 strong_typed_mention_match 0.730 0.730 0.730 strong_typed_all_match 0.749 0.711 0.730 strong_typed_link_match 0.674 0.797 0.731 strong_typed_nil_match 0.694 0.698 0.696 entity_match 0.876 0.876 0.876 mention_ceaf 0.808 0.808 0.808 typed_mention_ceaf 0.713 0.713 0.713 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 1.000 1.000 1.000 strong_mention_match 0.861 0.861 0.861 strong_typed_mention_match 0.686 0.686 0.686 strong_typed_all_match 0.651 0.665 0.658 strong_typed_link_match 0.780 0.739 0.759 strong_typed_nil_match 0.627 0.739 0.678 entity_match 0.805 0.805 0.805 mention_ceaf 0.744 0.744 0.744 typed_mention_ceaf 0.637 0.637 0.637 typed_mention_ceaf_plus ************************************************************* Run ID: RPI_BLENDER3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.986 1.000 0.993 strong_mention_match 0.868 0.880 0.874 strong_typed_mention_match 0.684 0.694 0.689 strong_typed_all_match 0.676 0.679 0.677 strong_typed_link_match 0.706 0.735 0.720 strong_typed_nil_match 0.642 0.751 0.692 entity_match 0.788 0.799 0.794 mention_ceaf 0.725 0.735 0.730 typed_mention_ceaf 0.643 0.652 0.648 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.971 1.000 0.985 strong_mention_match 0.855 0.880 0.867 strong_typed_mention_match 0.651 0.670 0.660 strong_typed_all_match 0.632 0.658 0.645 strong_typed_link_match 0.699 0.700 0.700 strong_typed_nil_match 0.621 0.793 0.697 entity_match 0.769 0.792 0.781 mention_ceaf 0.691 0.712 0.702 typed_mention_ceaf 0.597 0.615 0.606 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 1.000 1.000 1.000 strong_mention_match 0.891 0.891 0.891 strong_typed_mention_match 0.730 0.730 0.730 strong_typed_all_match 0.749 0.711 0.730 strong_typed_link_match 0.674 0.797 0.731 strong_typed_nil_match 0.694 0.698 0.696 entity_match 0.876 0.876 0.876 mention_ceaf 0.808 0.808 0.808 typed_mention_ceaf 0.713 0.713 0.713 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 1.000 1.000 1.000 strong_mention_match 0.861 0.861 0.861 strong_typed_mention_match 0.686 0.686 0.686 strong_typed_all_match 0.651 0.665 0.658 strong_typed_link_match 0.780 0.739 0.759 strong_typed_nil_match 0.627 0.739 0.678 entity_match 0.805 0.805 0.805 mention_ceaf 0.744 0.744 0.744 typed_mention_ceaf 0.637 0.637 0.637 typed_mention_ceaf_plus