======================================================================== TAC KBP 2017 TRI-LINGUAL ENTITY DISCOVERY AND LINKING EVALUATION RESULTS ======================================================================== Team ID: Boun Organization: Boğaziçi University ************************************************************* Run ID: Boun1 Languages addressed: English Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.817 0.144 0.245 strong_mention_match 0.691 0.122 0.207 strong_typed_mention_match 0.644 0.114 0.193 strong_typed_all_match 0.638 0.091 0.159 strong_typed_link_match 0.652 0.168 0.267 strong_typed_nil_match 0.694 0.125 0.212 entity_match 0.659 0.116 0.198 mention_ceaf 0.580 0.102 0.174 typed_mention_ceaf 0.552 0.097 0.165 typed_mention_ceaf_plus 0.718 0.069 0.126 b_cubed 0.631 0.062 0.113 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.817 0.507 0.626 strong_mention_match 0.691 0.429 0.529 strong_typed_mention_match 0.644 0.400 0.494 strong_typed_all_match 0.638 0.343 0.446 strong_typed_link_match 0.652 0.511 0.573 strong_typed_nil_match 0.694 0.433 0.533 entity_match 0.659 0.410 0.505 mention_ceaf 0.580 0.360 0.444 typed_mention_ceaf 0.552 0.343 0.423 typed_mention_ceaf_plus 0.718 0.316 0.439 b_cubed 0.631 0.287 0.395 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ************************************************************* Run ID: Boun2 Languages addressed: English Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.815 0.144 0.245 strong_mention_match 0.687 0.122 0.207 strong_typed_mention_match 0.638 0.113 0.192 strong_typed_all_match 0.629 0.090 0.158 strong_typed_link_match 0.651 0.168 0.267 strong_typed_nil_match 0.688 0.124 0.211 entity_match 0.654 0.116 0.197 mention_ceaf 0.572 0.101 0.172 typed_mention_ceaf 0.543 0.096 0.163 typed_mention_ceaf_plus 0.717 0.069 0.127 b_cubed 0.627 0.062 0.113 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.815 0.509 0.626 strong_mention_match 0.687 0.429 0.528 strong_typed_mention_match 0.638 0.398 0.490 strong_typed_all_match 0.629 0.340 0.442 strong_typed_link_match 0.651 0.511 0.573 strong_typed_nil_match 0.688 0.432 0.530 entity_match 0.654 0.408 0.503 mention_ceaf 0.572 0.357 0.440 typed_mention_ceaf 0.543 0.339 0.417 typed_mention_ceaf_plus 0.717 0.316 0.439 b_cubed 0.627 0.286 0.393 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ************************************************************* Run ID: Boun3 Languages addressed: English Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.822 0.140 0.239 strong_mention_match 0.696 0.119 0.203 strong_typed_mention_match 0.651 0.111 0.189 strong_typed_all_match 0.624 0.089 0.156 strong_typed_link_match 0.691 0.164 0.265 strong_typed_nil_match 0.678 0.127 0.214 entity_match 0.667 0.114 0.194 mention_ceaf 0.579 0.099 0.169 typed_mention_ceaf 0.555 0.095 0.162 typed_mention_ceaf_plus 0.752 0.067 0.123 b_cubed 0.664 0.061 0.111 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.822 0.494 0.617 strong_mention_match 0.696 0.418 0.522 strong_typed_mention_match 0.651 0.391 0.488 strong_typed_all_match 0.624 0.335 0.436 strong_typed_link_match 0.691 0.500 0.580 strong_typed_nil_match 0.678 0.441 0.534 entity_match 0.667 0.400 0.500 mention_ceaf 0.579 0.348 0.434 typed_mention_ceaf 0.555 0.333 0.417 typed_mention_ceaf_plus 0.752 0.307 0.436 b_cubed 0.664 0.281 0.395 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ************************************************************* Run ID: Boun4 Languages addressed: English Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.833 0.137 0.235 strong_mention_match 0.716 0.117 0.202 strong_typed_mention_match 0.671 0.110 0.189 strong_typed_all_match 0.625 0.088 0.154 strong_typed_link_match 0.741 0.163 0.268 strong_typed_nil_match 0.683 0.128 0.216 entity_match 0.692 0.113 0.195 mention_ceaf 0.605 0.099 0.171 typed_mention_ceaf 0.579 0.095 0.163 typed_mention_ceaf_plus 0.800 0.066 0.122 b_cubed 0.713 0.060 0.111 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.833 0.482 0.611 strong_mention_match 0.716 0.414 0.525 strong_typed_mention_match 0.671 0.388 0.491 strong_typed_all_match 0.625 0.331 0.433 strong_typed_link_match 0.741 0.499 0.596 strong_typed_nil_match 0.683 0.446 0.539 entity_match 0.692 0.400 0.507 mention_ceaf 0.605 0.350 0.444 typed_mention_ceaf 0.579 0.335 0.425 typed_mention_ceaf_plus 0.800 0.303 0.439 b_cubed 0.713 0.277 0.399 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ************************************************************* Run ID: Boun5 Languages addressed: English Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.658 0.084 0.150 strong_mention_match 0.405 0.052 0.092 strong_typed_mention_match 0.321 0.041 0.073 strong_typed_all_match 0.474 0.053 0.095 strong_typed_link_match 0.077 0.013 0.022 strong_typed_nil_match 0.575 0.086 0.150 entity_match 0.508 0.065 0.115 mention_ceaf 0.359 0.046 0.082 typed_mention_ceaf 0.318 0.041 0.072 typed_mention_ceaf_plus 0.613 0.028 0.053 b_cubed 0.418 0.021 0.040 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.658 0.297 0.410 strong_mention_match 0.405 0.183 0.252 strong_typed_mention_match 0.321 0.145 0.200 strong_typed_all_match 0.474 0.200 0.281 strong_typed_link_match 0.077 0.039 0.052 strong_typed_nil_match 0.575 0.300 0.394 entity_match 0.508 0.230 0.316 mention_ceaf 0.359 0.162 0.223 typed_mention_ceaf 0.318 0.144 0.198 typed_mention_ceaf_plus 0.613 0.138 0.225 b_cubed 0.418 0.107 0.171 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus