=============================================================================== TAC KBP 2016 ENTITY DISCOVERY AND LINKING EVALUATION RESULTS (FIRST EDL WINDOW) =============================================================================== Team ID: IBM Organization: IBM Research ************************************************************* Run ID: IBM1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.830 0.599 0.696 strong_typed_mention_match 0.649 0.469 0.544 strong_typed_all_match 0.729 0.432 0.543 strong_typed_link_match 0.508 0.596 0.548 strong_typed_nil_match 0.782 0.554 0.649 entity_match 0.717 0.517 0.601 mention_ceaf 0.692 0.500 0.580 typed_mention_ceaf 0.645 0.466 0.541 typed_mention_ceaf_plus 0.764 0.381 0.509 b_cubed 0.620 0.341 0.440 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.920 0.687 0.787 strong_mention_match 0.879 0.657 0.752 strong_typed_mention_match 0.659 0.492 0.563 strong_typed_all_match 0.708 0.458 0.556 strong_typed_link_match 0.555 0.615 0.583 strong_typed_nil_match 0.730 0.575 0.643 entity_match 0.759 0.567 0.649 mention_ceaf 0.734 0.548 0.628 typed_mention_ceaf 0.651 0.486 0.557 typed_mention_ceaf_plus 0.823 0.451 0.583 b_cubed 0.640 0.382 0.478 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.796 0.566 0.662 strong_mention_match 0.761 0.541 0.633 strong_typed_mention_match 0.622 0.442 0.517 strong_typed_all_match 0.746 0.413 0.532 strong_typed_link_match 0.413 0.561 0.476 strong_typed_nil_match 0.785 0.529 0.632 entity_match 0.704 0.501 0.585 mention_ceaf 0.680 0.484 0.565 typed_mention_ceaf 0.620 0.441 0.515 typed_mention_ceaf_plus 0.721 0.371 0.490 b_cubed 0.597 0.329 0.424 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.893 0.628 0.737 strong_mention_match 0.847 0.595 0.699 strong_typed_mention_match 0.671 0.472 0.554 strong_typed_all_match 0.740 0.423 0.538 strong_typed_link_match 0.568 0.608 0.587 strong_typed_nil_match 0.857 0.555 0.674 entity_match 0.761 0.535 0.628 mention_ceaf 0.731 0.514 0.603 typed_mention_ceaf 0.669 0.470 0.552 typed_mention_ceaf_plus 0.770 0.415 0.540 b_cubed 0.640 0.364 0.464 b_cubed_plus ************************************************************* Run ID: IBM2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.830 0.599 0.696 strong_typed_mention_match 0.664 0.480 0.557 strong_typed_all_match 0.731 0.446 0.554 strong_typed_link_match 0.537 0.596 0.565 strong_typed_nil_match 0.776 0.562 0.652 entity_match 0.733 0.529 0.615 mention_ceaf 0.708 0.511 0.593 typed_mention_ceaf 0.660 0.477 0.554 typed_mention_ceaf_plus 0.772 0.393 0.521 b_cubed 0.638 0.354 0.455 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.920 0.687 0.787 strong_mention_match 0.879 0.657 0.752 strong_typed_mention_match 0.658 0.492 0.563 strong_typed_all_match 0.707 0.457 0.555 strong_typed_link_match 0.556 0.615 0.584 strong_typed_nil_match 0.728 0.574 0.642 entity_match 0.758 0.566 0.648 mention_ceaf 0.733 0.548 0.627 typed_mention_ceaf 0.650 0.486 0.556 typed_mention_ceaf_plus 0.823 0.451 0.582 b_cubed 0.640 0.381 0.478 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.796 0.566 0.662 strong_mention_match 0.761 0.541 0.633 strong_typed_mention_match 0.666 0.474 0.554 strong_typed_all_match 0.749 0.452 0.564 strong_typed_link_match 0.487 0.561 0.521 strong_typed_nil_match 0.767 0.556 0.644 entity_match 0.753 0.536 0.626 mention_ceaf 0.725 0.516 0.603 typed_mention_ceaf 0.665 0.473 0.553 typed_mention_ceaf_plus 0.744 0.404 0.523 b_cubed 0.647 0.366 0.468 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.893 0.628 0.737 strong_mention_match 0.847 0.595 0.699 strong_typed_mention_match 0.671 0.472 0.554 strong_typed_all_match 0.740 0.423 0.539 strong_typed_link_match 0.568 0.608 0.587 strong_typed_nil_match 0.858 0.555 0.674 entity_match 0.761 0.535 0.628 mention_ceaf 0.731 0.514 0.603 typed_mention_ceaf 0.669 0.470 0.552 typed_mention_ceaf_plus 0.770 0.415 0.540 b_cubed 0.641 0.364 0.465 b_cubed_plus ************************************************************* Run ID: IBM3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.829 0.602 0.697 strong_typed_mention_match 0.646 0.469 0.544 strong_typed_all_match 0.727 0.431 0.541 strong_typed_link_match 0.505 0.601 0.549 strong_typed_nil_match 0.786 0.554 0.650 entity_match 0.712 0.517 0.599 mention_ceaf 0.687 0.499 0.578 typed_mention_ceaf 0.641 0.466 0.540 typed_mention_ceaf_plus 0.762 0.382 0.509 b_cubed 0.616 0.342 0.440 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.917 0.695 0.791 strong_mention_match 0.877 0.665 0.756 strong_typed_mention_match 0.650 0.493 0.561 strong_typed_all_match 0.703 0.455 0.552 strong_typed_link_match 0.545 0.628 0.583 strong_typed_nil_match 0.737 0.575 0.646 entity_match 0.749 0.568 0.646 mention_ceaf 0.725 0.550 0.625 typed_mention_ceaf 0.642 0.487 0.553 typed_mention_ceaf_plus 0.815 0.455 0.584 b_cubed 0.630 0.384 0.477 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.796 0.566 0.662 strong_mention_match 0.761 0.541 0.633 strong_typed_mention_match 0.622 0.442 0.517 strong_typed_all_match 0.746 0.413 0.532 strong_typed_link_match 0.413 0.561 0.476 strong_typed_nil_match 0.785 0.529 0.632 entity_match 0.704 0.501 0.585 mention_ceaf 0.680 0.484 0.565 typed_mention_ceaf 0.620 0.441 0.515 typed_mention_ceaf_plus 0.721 0.371 0.490 b_cubed 0.597 0.329 0.424 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.893 0.628 0.737 strong_mention_match 0.847 0.595 0.699 strong_typed_mention_match 0.671 0.472 0.554 strong_typed_all_match 0.740 0.423 0.538 strong_typed_link_match 0.568 0.608 0.587 strong_typed_nil_match 0.857 0.555 0.674 entity_match 0.761 0.535 0.628 mention_ceaf 0.731 0.514 0.603 typed_mention_ceaf 0.669 0.470 0.552 typed_mention_ceaf_plus 0.770 0.415 0.540 b_cubed 0.640 0.364 0.464 b_cubed_plus