================================================================================ TAC KBP 2016 ENTITY DISCOVERY AND LINKING EVALUATION RESULTS (SECOND EDL WINDOW) ================================================================================ Team ID: YorkNRM Organization: York University ************************************************************* Run ID: YorkNRM1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No Did the run use any of the distributed EDL1 runs (from the first evaluation window): No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.819 0.639 0.718 strong_typed_mention_match 0.706 0.551 0.619 strong_typed_all_match 0.772 0.533 0.631 strong_typed_link_match 0.561 0.614 0.587 strong_typed_nil_match 0.820 0.616 0.703 entity_match 0.734 0.572 0.643 mention_ceaf 0.717 0.559 0.628 typed_mention_ceaf 0.681 0.531 0.597 typed_mention_ceaf_plus 0.706 0.468 0.563 b_cubed 0.618 0.435 0.510 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.878 0.715 0.788 strong_mention_match 0.836 0.680 0.750 strong_typed_mention_match 0.715 0.582 0.642 strong_typed_all_match 0.749 0.573 0.649 strong_typed_link_match 0.623 0.613 0.618 strong_typed_nil_match 0.813 0.633 0.712 entity_match 0.740 0.602 0.664 mention_ceaf 0.724 0.589 0.650 typed_mention_ceaf 0.687 0.559 0.616 typed_mention_ceaf_plus 0.705 0.524 0.601 b_cubed 0.618 0.479 0.540 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.822 0.651 0.727 strong_mention_match 0.789 0.625 0.698 strong_typed_mention_match 0.694 0.550 0.613 strong_typed_all_match 0.786 0.527 0.631 strong_typed_link_match 0.497 0.643 0.561 strong_typed_nil_match 0.840 0.583 0.688 entity_match 0.752 0.596 0.665 mention_ceaf 0.735 0.583 0.650 typed_mention_ceaf 0.676 0.536 0.598 typed_mention_ceaf_plus 0.677 0.501 0.576 b_cubed 0.603 0.451 0.516 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.871 0.628 0.730 strong_mention_match 0.835 0.602 0.700 strong_typed_mention_match 0.711 0.512 0.596 strong_typed_all_match 0.793 0.486 0.603 strong_typed_link_match 0.573 0.588 0.580 strong_typed_nil_match 0.810 0.628 0.708 entity_match 0.756 0.545 0.633 mention_ceaf 0.734 0.529 0.615 typed_mention_ceaf 0.680 0.490 0.570 typed_mention_ceaf_plus 0.773 0.444 0.564 b_cubed 0.657 0.406 0.501 b_cubed_plus ************************************************************* Run ID: YorkNRM2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No Did the run use any of the distributed EDL1 runs (from the first evaluation window): No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.810 0.647 0.719 strong_typed_mention_match 0.181 0.145 0.161 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.181 0.649 0.283 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.045 0.036 0.040 mention_ceaf 0.045 0.036 0.040 typed_mention_ceaf 0.002 0.001 0.001 typed_mention_ceaf_plus 0.008 0.557 0.015 b_cubed 0.000 0.139 0.000 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.874 0.729 0.795 strong_mention_match 0.830 0.693 0.755 strong_typed_mention_match 0.174 0.145 0.158 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.174 0.661 0.275 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.033 0.027 0.030 mention_ceaf 0.033 0.027 0.030 typed_mention_ceaf 0.001 0.001 0.001 typed_mention_ceaf_plus 0.007 0.634 0.015 b_cubed 0.000 0.141 0.000 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.805 0.672 0.733 strong_mention_match 0.767 0.640 0.698 strong_typed_mention_match 0.162 0.135 0.147 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.162 0.691 0.262 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.066 0.055 0.060 mention_ceaf 0.065 0.054 0.059 typed_mention_ceaf 0.004 0.004 0.004 typed_mention_ceaf_plus 0.013 0.557 0.025 b_cubed 0.000 0.131 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.878 0.621 0.727 strong_mention_match 0.843 0.596 0.698 strong_typed_mention_match 0.221 0.156 0.183 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.221 0.596 0.322 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.038 0.027 0.031 mention_ceaf 0.035 0.025 0.029 typed_mention_ceaf 0.006 0.004 0.005 typed_mention_ceaf_plus 0.007 0.505 0.014 b_cubed 0.000 0.151 0.000 b_cubed_plus ************************************************************* Run ID: YorkNRM3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No Did the run use any of the distributed EDL1 runs (from the first evaluation window): No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.774 0.735 0.754 strong_typed_mention_match 0.218 0.207 0.212 strong_typed_all_match 0.528 0.066 0.118 strong_typed_link_match 0.182 0.698 0.289 strong_typed_nil_match 0.721 0.183 0.292 entity_match 0.121 0.115 0.118 mention_ceaf 0.112 0.107 0.109 typed_mention_ceaf 0.065 0.062 0.064 typed_mention_ceaf_plus 0.093 0.591 0.161 b_cubed 0.059 0.163 0.087 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.854 0.802 0.827 strong_mention_match 0.804 0.755 0.779 strong_typed_mention_match 0.201 0.188 0.194 strong_typed_all_match 0.482 0.046 0.083 strong_typed_link_match 0.177 0.697 0.282 strong_typed_nil_match 0.721 0.149 0.247 entity_match 0.093 0.088 0.090 mention_ceaf 0.086 0.080 0.083 typed_mention_ceaf 0.046 0.044 0.045 typed_mention_ceaf_plus 0.072 0.663 0.130 b_cubed 0.046 0.161 0.072 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.782 0.808 0.795 strong_mention_match 0.735 0.760 0.747 strong_typed_mention_match 0.213 0.220 0.217 strong_typed_all_match 0.566 0.092 0.158 strong_typed_link_match 0.162 0.747 0.266 strong_typed_nil_match 0.753 0.254 0.380 entity_match 0.156 0.161 0.158 mention_ceaf 0.146 0.151 0.148 typed_mention_ceaf 0.085 0.088 0.087 typed_mention_ceaf_plus 0.117 0.611 0.197 b_cubed 0.073 0.168 0.102 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.833 0.715 0.770 strong_mention_match 0.790 0.678 0.730 strong_typed_mention_match 0.250 0.215 0.231 strong_typed_all_match 0.507 0.060 0.107 strong_typed_link_match 0.221 0.651 0.330 strong_typed_nil_match 0.671 0.153 0.249 entity_match 0.126 0.108 0.116 mention_ceaf 0.112 0.096 0.104 typed_mention_ceaf 0.069 0.059 0.064 typed_mention_ceaf_plus 0.103 0.545 0.173 b_cubed 0.066 0.178 0.096 b_cubed_plus