=============================================================================== TAC KBP 2016 ENTITY DISCOVERY AND LINKING EVALUATION RESULTS (FIRST EDL WINDOW) =============================================================================== Team ID: YorkNRM Organization: York University ************************************************************* Run ID: YorkNRM1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.830 0.241 0.373 strong_typed_mention_match 0.182 0.053 0.082 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.182 0.237 0.206 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.034 0.010 0.015 mention_ceaf 0.034 0.010 0.015 typed_mention_ceaf 0.002 0.000 0.001 typed_mention_ceaf_plus 0.008 0.141 0.014 b_cubed 0.000 0.051 0.000 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.870 0.684 0.766 strong_mention_match 0.830 0.652 0.731 strong_typed_mention_match 0.182 0.143 0.160 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.182 0.652 0.284 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.034 0.027 0.030 mention_ceaf 0.034 0.027 0.030 typed_mention_ceaf 0.002 0.001 0.001 typed_mention_ceaf_plus 0.008 0.578 0.015 b_cubed 0.000 0.141 0.000 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ************************************************************* Run ID: YorkNRM2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.855 0.223 0.353 strong_typed_mention_match 0.190 0.049 0.078 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.190 0.222 0.205 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.035 0.009 0.015 mention_ceaf 0.035 0.009 0.015 typed_mention_ceaf 0.002 0.000 0.001 typed_mention_ceaf_plus 0.008 0.125 0.015 b_cubed 0.000 0.047 0.000 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.895 0.633 0.742 strong_mention_match 0.855 0.604 0.708 strong_typed_mention_match 0.190 0.134 0.157 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.190 0.612 0.290 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.035 0.025 0.029 mention_ceaf 0.035 0.025 0.029 typed_mention_ceaf 0.002 0.001 0.001 typed_mention_ceaf_plus 0.008 0.514 0.016 b_cubed 0.000 0.129 0.000 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ************************************************************* Run ID: YorkNRM3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.818 0.221 0.348 strong_typed_mention_match 0.197 0.053 0.084 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.197 0.239 0.216 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.039 0.011 0.017 mention_ceaf 0.039 0.010 0.016 typed_mention_ceaf 0.001 0.000 0.001 typed_mention_ceaf_plus 0.007 0.124 0.013 b_cubed 0.000 0.051 0.000 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.859 0.630 0.727 strong_mention_match 0.818 0.600 0.693 strong_typed_mention_match 0.197 0.144 0.166 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.197 0.658 0.303 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.039 0.029 0.033 mention_ceaf 0.039 0.028 0.033 typed_mention_ceaf 0.001 0.001 0.001 typed_mention_ceaf_plus 0.007 0.512 0.014 b_cubed 0.000 0.138 0.000 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ************************************************************* Run ID: YorkNRM4 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.859 0.221 0.352 strong_typed_mention_match 0.198 0.051 0.081 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.198 0.229 0.212 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.039 0.010 0.016 mention_ceaf 0.039 0.010 0.016 typed_mention_ceaf 0.001 0.000 0.001 typed_mention_ceaf_plus 0.008 0.123 0.015 b_cubed 0.000 0.049 0.000 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.892 0.623 0.734 strong_mention_match 0.859 0.601 0.707 strong_typed_mention_match 0.198 0.138 0.163 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.198 0.630 0.301 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.039 0.027 0.032 mention_ceaf 0.039 0.027 0.032 typed_mention_ceaf 0.001 0.001 0.001 typed_mention_ceaf_plus 0.008 0.495 0.015 b_cubed 0.000 0.132 0.000 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus