=============================================================================== TAC KBP 2016 ENTITY DISCOVERY AND LINKING EVALUATION RESULTS (FIRST EDL WINDOW) =============================================================================== Team ID: REDES Organization: Universidad de JaƩn and Universidad de Alicante ************************************************************* Run ID: REDES1 Did the run access the live Web during the evaluation window: Yes Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.496 0.170 0.253 strong_typed_mention_match 0.230 0.079 0.118 strong_typed_all_match 0.239 0.100 0.141 strong_typed_link_match 0.077 0.006 0.012 strong_typed_nil_match 0.326 0.237 0.274 entity_match 0.530 0.182 0.270 mention_ceaf 0.363 0.124 0.185 typed_mention_ceaf 0.229 0.079 0.117 typed_mention_ceaf_plus 0.666 0.085 0.151 b_cubed 0.353 0.050 0.088 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.813 0.417 0.552 strong_mention_match 0.621 0.319 0.421 strong_typed_mention_match 0.287 0.147 0.195 strong_typed_all_match 0.291 0.187 0.227 strong_typed_link_match 0.134 0.007 0.014 strong_typed_nil_match 0.354 0.352 0.353 entity_match 0.650 0.333 0.441 mention_ceaf 0.492 0.252 0.333 typed_mention_ceaf 0.286 0.147 0.194 typed_mention_ceaf_plus 0.750 0.213 0.332 b_cubed 0.383 0.118 0.180 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.622 0.344 0.443 strong_mention_match 0.343 0.190 0.244 strong_typed_mention_match 0.160 0.088 0.114 strong_typed_all_match 0.170 0.116 0.138 strong_typed_link_match 0.058 0.011 0.018 strong_typed_nil_match 0.296 0.339 0.316 entity_match 0.518 0.286 0.369 mention_ceaf 0.290 0.160 0.207 typed_mention_ceaf 0.159 0.088 0.113 typed_mention_ceaf_plus 0.568 0.177 0.270 b_cubed 0.319 0.108 0.162 b_cubed_plus ************************************************************* Run ID: REDES2 Did the run access the live Web during the evaluation window: Yes Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.496 0.170 0.253 strong_typed_mention_match 0.231 0.079 0.118 strong_typed_all_match 0.240 0.100 0.141 strong_typed_link_match 0.077 0.006 0.012 strong_typed_nil_match 0.331 0.244 0.281 entity_match 0.525 0.180 0.268 mention_ceaf 0.360 0.123 0.184 typed_mention_ceaf 0.230 0.079 0.118 typed_mention_ceaf_plus 0.676 0.083 0.148 b_cubed 0.349 0.047 0.083 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.813 0.417 0.552 strong_mention_match 0.621 0.319 0.421 strong_typed_mention_match 0.287 0.147 0.195 strong_typed_all_match 0.291 0.187 0.227 strong_typed_link_match 0.134 0.007 0.014 strong_typed_nil_match 0.359 0.363 0.361 entity_match 0.651 0.334 0.441 mention_ceaf 0.491 0.252 0.333 typed_mention_ceaf 0.286 0.147 0.194 typed_mention_ceaf_plus 0.763 0.210 0.330 b_cubed 0.374 0.108 0.167 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.622 0.344 0.443 strong_mention_match 0.343 0.190 0.244 strong_typed_mention_match 0.162 0.090 0.116 strong_typed_all_match 0.173 0.118 0.140 strong_typed_link_match 0.058 0.011 0.018 strong_typed_nil_match 0.302 0.350 0.324 entity_match 0.517 0.286 0.368 mention_ceaf 0.290 0.160 0.206 typed_mention_ceaf 0.162 0.089 0.115 typed_mention_ceaf_plus 0.572 0.176 0.269 b_cubed 0.320 0.108 0.161 b_cubed_plus ************************************************************* Run ID: REDES3 Did the run access the live Web during the evaluation window: Yes Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.537 0.151 0.235 strong_typed_mention_match 0.254 0.071 0.111 strong_typed_all_match 0.256 0.092 0.135 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.290 0.197 0.235 entity_match 0.528 0.148 0.231 mention_ceaf 0.392 0.110 0.172 typed_mention_ceaf 0.254 0.071 0.111 typed_mention_ceaf_plus 0.667 0.061 0.111 b_cubed 0.322 0.032 0.058 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.722 0.419 0.530 strong_mention_match 0.534 0.309 0.392 strong_typed_mention_match 0.248 0.144 0.182 strong_typed_all_match 0.250 0.184 0.212 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.276 0.373 0.317 entity_match 0.560 0.324 0.411 mention_ceaf 0.413 0.240 0.303 typed_mention_ceaf 0.248 0.144 0.182 typed_mention_ceaf_plus 0.673 0.194 0.301 b_cubed 0.320 0.099 0.151 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.711 0.171 0.276 strong_mention_match 0.548 0.132 0.212 strong_typed_mention_match 0.272 0.065 0.105 strong_typed_all_match 0.272 0.089 0.134 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.331 0.186 0.238 entity_match 0.631 0.151 0.244 mention_ceaf 0.487 0.117 0.189 typed_mention_ceaf 0.272 0.065 0.105 typed_mention_ceaf_plus 0.659 0.081 0.145 b_cubed 0.330 0.046 0.080 b_cubed_plus ************************************************************* Run ID: REDES4 Did the run access the live Web during the evaluation window: Yes Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.537 0.151 0.235 strong_typed_mention_match 0.259 0.073 0.114 strong_typed_all_match 0.270 0.092 0.137 strong_typed_link_match 0.087 0.006 0.012 strong_typed_nil_match 0.310 0.196 0.240 entity_match 0.524 0.147 0.230 mention_ceaf 0.389 0.109 0.171 typed_mention_ceaf 0.258 0.072 0.113 typed_mention_ceaf_plus 0.672 0.060 0.111 b_cubed 0.329 0.032 0.058 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.722 0.419 0.530 strong_mention_match 0.534 0.309 0.392 strong_typed_mention_match 0.250 0.145 0.184 strong_typed_all_match 0.258 0.183 0.214 strong_typed_link_match 0.087 0.010 0.018 strong_typed_nil_match 0.288 0.370 0.324 entity_match 0.558 0.324 0.410 mention_ceaf 0.412 0.239 0.302 typed_mention_ceaf 0.249 0.144 0.183 typed_mention_ceaf_plus 0.676 0.194 0.301 b_cubed 0.322 0.099 0.152 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.000 0.000 0.000 strong_mention_match 0.000 0.000 0.000 strong_typed_mention_match 0.000 0.000 0.000 strong_typed_all_match 0.000 0.000 0.000 strong_typed_link_match 0.000 0.000 0.000 strong_typed_nil_match 0.000 0.000 0.000 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf_plus 0.000 0.000 0.000 b_cubed 0.000 0.000 0.000 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.711 0.171 0.276 strong_mention_match 0.548 0.132 0.212 strong_typed_mention_match 0.288 0.069 0.111 strong_typed_all_match 0.311 0.091 0.140 strong_typed_link_match 0.086 0.008 0.015 strong_typed_nil_match 0.380 0.185 0.249 entity_match 0.632 0.152 0.245 mention_ceaf 0.489 0.117 0.189 typed_mention_ceaf 0.286 0.069 0.111 typed_mention_ceaf_plus 0.671 0.081 0.145 b_cubed 0.356 0.048 0.085 b_cubed_plus