======================================================================= TAC KBP 2015 DIAGNOSTIC TRILINGUAL ENTITY LINKING EVALUATION RESULTS ======================================================================= Team ID: IBM Organization: IBM T.J. Watson Research Center ************************************************************* Run ID: IBM1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.986 0.987 0.987 strong_mention_match 0.858 0.859 0.859 strong_typed_mention_match 0.741 0.742 0.742 strong_typed_all_match 0.761 0.728 0.744 strong_typed_link_match 0.695 0.784 0.737 strong_typed_nil_match 0.771 0.789 0.780 entity_match 0.829 0.830 0.829 mention_ceaf 0.750 0.751 0.751 typed_mention_ceaf 0.723 0.724 0.724 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.970 0.973 0.972 strong_mention_match 0.868 0.871 0.870 strong_typed_mention_match 0.736 0.738 0.737 strong_typed_all_match 0.736 0.733 0.734 strong_typed_link_match 0.736 0.750 0.743 strong_typed_nil_match 0.715 0.784 0.748 entity_match 0.799 0.801 0.800 mention_ceaf 0.742 0.744 0.743 typed_mention_ceaf 0.705 0.707 0.706 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 1.000 1.000 1.000 strong_mention_match 0.828 0.828 0.828 strong_typed_mention_match 0.716 0.716 0.716 strong_typed_all_match 0.773 0.698 0.734 strong_typed_link_match 0.579 0.782 0.665 strong_typed_nil_match 0.806 0.762 0.783 entity_match 0.875 0.875 0.875 mention_ceaf 0.748 0.748 0.748 typed_mention_ceaf 0.713 0.713 0.713 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 1.000 1.000 1.000 strong_mention_match 0.889 0.889 0.889 strong_typed_mention_match 0.804 0.804 0.804 strong_typed_all_match 0.804 0.776 0.790 strong_typed_link_match 0.805 0.875 0.838 strong_typed_nil_match 0.842 0.835 0.838 entity_match 0.912 0.912 0.912 mention_ceaf 0.833 0.833 0.833 typed_mention_ceaf 0.796 0.796 0.796 typed_mention_ceaf_plus ************************************************************* Run ID: IBM2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.986 0.987 0.987 strong_mention_match 0.858 0.859 0.859 strong_typed_mention_match 0.741 0.742 0.742 strong_typed_all_match 0.760 0.728 0.744 strong_typed_link_match 0.695 0.784 0.737 strong_typed_nil_match 0.770 0.789 0.780 entity_match 0.829 0.830 0.829 mention_ceaf 0.750 0.751 0.751 typed_mention_ceaf 0.723 0.724 0.724 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.970 0.973 0.972 strong_mention_match 0.868 0.871 0.869 strong_typed_mention_match 0.736 0.738 0.737 strong_typed_all_match 0.735 0.733 0.734 strong_typed_link_match 0.736 0.750 0.743 strong_typed_nil_match 0.715 0.784 0.748 entity_match 0.799 0.801 0.800 mention_ceaf 0.742 0.744 0.743 typed_mention_ceaf 0.705 0.707 0.706 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 1.000 1.000 1.000 strong_mention_match 0.828 0.828 0.828 strong_typed_mention_match 0.716 0.716 0.716 strong_typed_all_match 0.773 0.698 0.733 strong_typed_link_match 0.579 0.782 0.665 strong_typed_nil_match 0.806 0.762 0.783 entity_match 0.875 0.875 0.875 mention_ceaf 0.748 0.748 0.748 typed_mention_ceaf 0.713 0.713 0.713 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 1.000 1.000 1.000 strong_mention_match 0.889 0.889 0.889 strong_typed_mention_match 0.804 0.804 0.804 strong_typed_all_match 0.803 0.776 0.790 strong_typed_link_match 0.805 0.875 0.838 strong_typed_nil_match 0.841 0.835 0.838 entity_match 0.912 0.912 0.912 mention_ceaf 0.833 0.833 0.833 typed_mention_ceaf 0.796 0.796 0.796 typed_mention_ceaf_plus ************************************************************* Run ID: IBM3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.986 0.987 0.987 strong_mention_match 0.858 0.860 0.859 strong_typed_mention_match 0.741 0.742 0.742 strong_typed_all_match 0.760 0.728 0.744 strong_typed_link_match 0.697 0.783 0.737 strong_typed_nil_match 0.770 0.789 0.779 entity_match 0.829 0.830 0.829 mention_ceaf 0.750 0.751 0.751 typed_mention_ceaf 0.723 0.724 0.723 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.970 0.973 0.972 strong_mention_match 0.868 0.871 0.870 strong_typed_mention_match 0.736 0.738 0.737 strong_typed_all_match 0.735 0.733 0.734 strong_typed_link_match 0.739 0.751 0.745 strong_typed_nil_match 0.716 0.784 0.748 entity_match 0.799 0.801 0.800 mention_ceaf 0.741 0.744 0.743 typed_mention_ceaf 0.704 0.706 0.705 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 1.000 1.000 1.000 strong_mention_match 0.829 0.829 0.829 strong_typed_mention_match 0.716 0.716 0.716 strong_typed_all_match 0.772 0.698 0.733 strong_typed_link_match 0.580 0.779 0.665 strong_typed_nil_match 0.802 0.761 0.781 entity_match 0.875 0.875 0.875 mention_ceaf 0.748 0.748 0.748 typed_mention_ceaf 0.713 0.713 0.713 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 1.000 1.000 1.000 strong_mention_match 0.889 0.889 0.889 strong_typed_mention_match 0.804 0.804 0.804 strong_typed_all_match 0.803 0.776 0.790 strong_typed_link_match 0.805 0.875 0.838 strong_typed_nil_match 0.842 0.835 0.838 entity_match 0.912 0.912 0.912 mention_ceaf 0.833 0.833 0.833 typed_mention_ceaf 0.796 0.796 0.796 typed_mention_ceaf_plus