======================================================================= TAC KBP 2015 TRILINGUAL ENTITY DISCOVERY AND LINKING EVALUATION RESULTS ======================================================================= Team ID: OSU Organization: Oregon State University ************************************************************* Run ID: OSU1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.727 0.553 0.628 strong_mention_match 0.642 0.489 0.556 strong_typed_mention_match 0.442 0.336 0.382 strong_typed_all_match 0.457 0.358 0.401 strong_typed_link_match 0.394 0.275 0.324 strong_typed_nil_match 0.455 0.483 0.469 entity_match 0.547 0.416 0.473 mention_ceaf 0.495 0.377 0.428 typed_mention_ceaf 0.428 0.326 0.370 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.771 0.691 0.729 strong_mention_match 0.674 0.605 0.637 strong_typed_mention_match 0.501 0.450 0.474 strong_typed_all_match 0.447 0.439 0.443 strong_typed_link_match 0.694 0.477 0.566 strong_typed_nil_match 0.482 0.629 0.546 entity_match 0.636 0.571 0.602 mention_ceaf 0.569 0.510 0.538 typed_mention_ceaf 0.492 0.441 0.465 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.762 0.369 0.497 strong_mention_match 0.720 0.349 0.470 strong_typed_mention_match 0.458 0.222 0.299 strong_typed_all_match 0.686 0.266 0.384 strong_typed_link_match 0.076 0.063 0.069 strong_typed_nil_match 0.652 0.284 0.396 entity_match 0.528 0.256 0.344 mention_ceaf 0.506 0.245 0.330 typed_mention_ceaf 0.436 0.211 0.285 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.582 0.543 0.562 strong_mention_match 0.488 0.456 0.471 strong_typed_mention_match 0.276 0.258 0.266 strong_typed_all_match 0.310 0.337 0.323 strong_typed_link_match 0.099 0.053 0.069 strong_typed_nil_match 0.329 0.467 0.386 entity_match 0.470 0.439 0.454 mention_ceaf 0.402 0.375 0.388 typed_mention_ceaf 0.275 0.257 0.266 typed_mention_ceaf_plus ************************************************************* Run ID: OSU2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.727 0.553 0.628 strong_mention_match 0.602 0.459 0.521 strong_typed_mention_match 0.415 0.316 0.359 strong_typed_all_match 0.422 0.331 0.371 strong_typed_link_match 0.393 0.275 0.323 strong_typed_nil_match 0.455 0.483 0.469 entity_match 0.547 0.416 0.473 mention_ceaf 0.465 0.354 0.402 typed_mention_ceaf 0.402 0.306 0.347 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.771 0.691 0.729 strong_mention_match 0.674 0.605 0.637 strong_typed_mention_match 0.501 0.450 0.474 strong_typed_all_match 0.447 0.439 0.443 strong_typed_link_match 0.694 0.477 0.566 strong_typed_nil_match 0.482 0.629 0.546 entity_match 0.636 0.571 0.602 mention_ceaf 0.569 0.510 0.538 typed_mention_ceaf 0.492 0.441 0.465 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.762 0.369 0.497 strong_mention_match 0.720 0.349 0.470 strong_typed_mention_match 0.458 0.222 0.299 strong_typed_all_match 0.686 0.266 0.384 strong_typed_link_match 0.076 0.063 0.069 strong_typed_nil_match 0.652 0.284 0.396 entity_match 0.528 0.256 0.344 mention_ceaf 0.506 0.245 0.330 typed_mention_ceaf 0.436 0.211 0.285 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.582 0.543 0.562 strong_mention_match 0.307 0.287 0.296 strong_typed_mention_match 0.156 0.146 0.151 strong_typed_all_match 0.168 0.183 0.176 strong_typed_link_match 0.091 0.049 0.064 strong_typed_nil_match 0.329 0.467 0.386 entity_match 0.470 0.439 0.454 mention_ceaf 0.247 0.231 0.239 typed_mention_ceaf 0.156 0.145 0.150 typed_mention_ceaf_plus ************************************************************* Run ID: OSU3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.661 0.577 0.616 strong_mention_match 0.585 0.511 0.546 strong_typed_mention_match 0.401 0.350 0.374 strong_typed_all_match 0.457 0.358 0.401 strong_typed_link_match 0.292 0.329 0.310 strong_typed_nil_match 0.455 0.483 0.469 entity_match 0.482 0.421 0.449 mention_ceaf 0.437 0.382 0.407 typed_mention_ceaf 0.379 0.331 0.353 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.654 0.741 0.695 strong_mention_match 0.575 0.651 0.611 strong_typed_mention_match 0.423 0.479 0.449 strong_typed_all_match 0.447 0.439 0.443 strong_typed_link_match 0.383 0.581 0.462 strong_typed_nil_match 0.482 0.629 0.546 entity_match 0.512 0.580 0.544 mention_ceaf 0.459 0.520 0.488 typed_mention_ceaf 0.398 0.452 0.423 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.762 0.369 0.497 strong_mention_match 0.720 0.349 0.470 strong_typed_mention_match 0.458 0.222 0.299 strong_typed_all_match 0.686 0.266 0.384 strong_typed_link_match 0.076 0.063 0.069 strong_typed_nil_match 0.652 0.284 0.396 entity_match 0.528 0.256 0.344 mention_ceaf 0.506 0.245 0.330 typed_mention_ceaf 0.436 0.211 0.285 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.582 0.543 0.562 strong_mention_match 0.488 0.456 0.471 strong_typed_mention_match 0.276 0.258 0.266 strong_typed_all_match 0.310 0.337 0.323 strong_typed_link_match 0.099 0.053 0.069 strong_typed_nil_match 0.329 0.467 0.386 entity_match 0.470 0.439 0.454 mention_ceaf 0.402 0.375 0.388 typed_mention_ceaf 0.275 0.257 0.266 typed_mention_ceaf_plus ************************************************************* Run ID: OSU4 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.661 0.577 0.616 strong_mention_match 0.550 0.481 0.513 strong_typed_mention_match 0.378 0.330 0.352 strong_typed_all_match 0.422 0.331 0.371 strong_typed_link_match 0.291 0.328 0.309 strong_typed_nil_match 0.455 0.483 0.469 entity_match 0.482 0.421 0.449 mention_ceaf 0.411 0.359 0.383 typed_mention_ceaf 0.356 0.311 0.332 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.654 0.741 0.695 strong_mention_match 0.575 0.651 0.611 strong_typed_mention_match 0.423 0.479 0.449 strong_typed_all_match 0.447 0.439 0.443 strong_typed_link_match 0.383 0.581 0.462 strong_typed_nil_match 0.482 0.629 0.546 entity_match 0.512 0.580 0.544 mention_ceaf 0.459 0.520 0.488 typed_mention_ceaf 0.398 0.452 0.423 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.762 0.369 0.497 strong_mention_match 0.720 0.349 0.470 strong_typed_mention_match 0.458 0.222 0.299 strong_typed_all_match 0.686 0.266 0.384 strong_typed_link_match 0.076 0.063 0.069 strong_typed_nil_match 0.652 0.284 0.396 entity_match 0.528 0.256 0.344 mention_ceaf 0.506 0.245 0.330 typed_mention_ceaf 0.436 0.211 0.285 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.582 0.543 0.562 strong_mention_match 0.307 0.287 0.296 strong_typed_mention_match 0.156 0.146 0.151 strong_typed_all_match 0.168 0.183 0.176 strong_typed_link_match 0.091 0.049 0.064 strong_typed_nil_match 0.329 0.467 0.386 entity_match 0.470 0.439 0.454 mention_ceaf 0.247 0.231 0.239 typed_mention_ceaf 0.156 0.145 0.150 typed_mention_ceaf_plus