======================================================================== TAC KBP 2017 TRI-LINGUAL ENTITY DISCOVERY AND LINKING EVALUATION RESULTS ======================================================================== Team ID: UI_CCG Organization: University of Illinois at Urbana Champaign ************************************************************* Run ID: UI_CCG1 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.885 0.677 0.767 strong_mention_match 0.852 0.653 0.739 strong_typed_mention_match 0.660 0.505 0.572 strong_typed_all_match 0.641 0.509 0.567 strong_typed_link_match 0.712 0.495 0.584 strong_typed_nil_match 0.586 0.632 0.608 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.558 0.427 0.484 typed_mention_ceaf_plus 0.637 0.427 0.511 b_cubed 0.528 0.368 0.434 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.896 0.757 0.820 strong_mention_match 0.861 0.727 0.789 strong_typed_mention_match 0.641 0.542 0.587 strong_typed_all_match 0.598 0.535 0.565 strong_typed_link_match 0.744 0.555 0.636 strong_typed_nil_match 0.538 0.667 0.596 entity_match 0.684 0.578 0.627 mention_ceaf 0.660 0.557 0.604 typed_mention_ceaf 0.573 0.484 0.525 typed_mention_ceaf_plus 0.786 0.494 0.607 b_cubed 0.631 0.403 0.492 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.866 0.598 0.708 strong_mention_match 0.841 0.581 0.687 strong_typed_mention_match 0.684 0.473 0.559 strong_typed_all_match 0.690 0.512 0.588 strong_typed_link_match 0.661 0.356 0.463 strong_typed_nil_match 0.646 0.632 0.639 entity_match 0.733 0.507 0.599 mention_ceaf 0.715 0.494 0.585 typed_mention_ceaf 0.661 0.456 0.540 typed_mention_ceaf_plus 0.767 0.404 0.529 b_cubed 0.662 0.354 0.461 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.896 0.714 0.795 strong_mention_match 0.857 0.683 0.760 strong_typed_mention_match 0.648 0.516 0.575 strong_typed_all_match 0.614 0.482 0.540 strong_typed_link_match 0.720 0.592 0.650 strong_typed_nil_match 0.574 0.603 0.588 entity_match 0.697 0.555 0.618 mention_ceaf 0.673 0.536 0.596 typed_mention_ceaf 0.607 0.484 0.538 typed_mention_ceaf_plus 0.767 0.461 0.576 b_cubed 0.634 0.393 0.485 b_cubed_plus ************************************************************* Run ID: UI_CCG2 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.917 0.584 0.714 strong_mention_match 0.886 0.564 0.689 strong_typed_mention_match 0.758 0.483 0.590 strong_typed_all_match 0.744 0.500 0.598 strong_typed_link_match 0.798 0.439 0.567 strong_typed_nil_match 0.721 0.630 0.673 entity_match 0.717 0.457 0.558 mention_ceaf 0.695 0.443 0.541 typed_mention_ceaf 0.647 0.412 0.503 typed_mention_ceaf_plus 0.737 0.384 0.505 b_cubed 0.631 0.350 0.450 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.942 0.621 0.749 strong_mention_match 0.908 0.599 0.722 strong_typed_mention_match 0.764 0.504 0.608 strong_typed_all_match 0.731 0.522 0.609 strong_typed_link_match 0.847 0.469 0.604 strong_typed_nil_match 0.710 0.665 0.687 entity_match 0.798 0.527 0.634 mention_ceaf 0.771 0.509 0.613 typed_mention_ceaf 0.690 0.456 0.549 typed_mention_ceaf_plus 0.908 0.424 0.578 b_cubed 0.768 0.371 0.501 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.893 0.557 0.686 strong_mention_match 0.868 0.542 0.667 strong_typed_mention_match 0.743 0.463 0.571 strong_typed_all_match 0.746 0.509 0.605 strong_typed_link_match 0.726 0.327 0.451 strong_typed_nil_match 0.718 0.630 0.671 entity_match 0.789 0.493 0.607 mention_ceaf 0.771 0.481 0.593 typed_mention_ceaf 0.719 0.449 0.553 typed_mention_ceaf_plus 0.830 0.384 0.525 b_cubed 0.728 0.347 0.470 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.927 0.587 0.718 strong_mention_match 0.887 0.561 0.688 strong_typed_mention_match 0.773 0.489 0.599 strong_typed_all_match 0.753 0.467 0.577 strong_typed_link_match 0.813 0.537 0.647 strong_typed_nil_match 0.736 0.601 0.661 entity_match 0.812 0.513 0.629 mention_ceaf 0.783 0.495 0.607 typed_mention_ceaf 0.729 0.461 0.565 typed_mention_ceaf_plus 0.887 0.403 0.554 b_cubed 0.772 0.370 0.500 b_cubed_plus ************************************************************* Run ID: UI_CCG3 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.863 0.692 0.768 strong_mention_match 0.831 0.666 0.739 strong_typed_mention_match 0.630 0.505 0.561 strong_typed_all_match 0.613 0.506 0.554 strong_typed_link_match 0.677 0.503 0.577 strong_typed_nil_match 0.565 0.628 0.595 entity_match 0.000 0.000 0.000 mention_ceaf 0.000 0.000 0.000 typed_mention_ceaf 0.530 0.424 0.471 typed_mention_ceaf_plus 0.605 0.431 0.503 b_cubed 0.497 0.367 0.422 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.885 0.763 0.820 strong_mention_match 0.851 0.733 0.788 strong_typed_mention_match 0.628 0.541 0.582 strong_typed_all_match 0.586 0.532 0.558 strong_typed_link_match 0.727 0.558 0.632 strong_typed_nil_match 0.525 0.661 0.585 entity_match 0.672 0.579 0.622 mention_ceaf 0.648 0.558 0.600 typed_mention_ceaf 0.561 0.483 0.519 typed_mention_ceaf_plus 0.770 0.499 0.605 b_cubed 0.617 0.403 0.488 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.827 0.619 0.708 strong_mention_match 0.803 0.601 0.687 strong_typed_mention_match 0.635 0.475 0.543 strong_typed_all_match 0.647 0.510 0.570 strong_typed_link_match 0.589 0.371 0.455 strong_typed_nil_match 0.614 0.633 0.623 entity_match 0.683 0.510 0.584 mention_ceaf 0.666 0.498 0.569 typed_mention_ceaf 0.612 0.458 0.524 typed_mention_ceaf_plus 0.707 0.411 0.520 b_cubed 0.606 0.355 0.448 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.888 0.728 0.800 strong_mention_match 0.846 0.694 0.763 strong_typed_mention_match 0.626 0.513 0.564 strong_typed_all_match 0.589 0.476 0.527 strong_typed_link_match 0.705 0.596 0.646 strong_typed_nil_match 0.555 0.596 0.575 entity_match 0.681 0.558 0.614 mention_ceaf 0.656 0.538 0.591 typed_mention_ceaf 0.586 0.481 0.528 typed_mention_ceaf_plus 0.746 0.467 0.574 b_cubed 0.608 0.391 0.476 b_cubed_plus ************************************************************* Run ID: UI_CCG4 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.885 0.677 0.767 strong_mention_match 0.852 0.653 0.739 strong_typed_mention_match 0.684 0.524 0.593 strong_typed_all_match 0.674 0.536 0.597 strong_typed_link_match 0.712 0.495 0.584 strong_typed_nil_match 0.762 0.631 0.690 entity_match 0.656 0.503 0.569 mention_ceaf 0.637 0.488 0.552 typed_mention_ceaf 0.583 0.446 0.505 typed_mention_ceaf_plus 0.608 0.454 0.520 b_cubed 0.522 0.392 0.448 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.896 0.757 0.820 strong_mention_match 0.861 0.727 0.789 strong_typed_mention_match 0.677 0.572 0.620 strong_typed_all_match 0.648 0.580 0.612 strong_typed_link_match 0.744 0.555 0.636 strong_typed_nil_match 0.746 0.680 0.711 entity_match 0.712 0.601 0.652 mention_ceaf 0.690 0.582 0.631 typed_mention_ceaf 0.608 0.514 0.557 typed_mention_ceaf_plus 0.748 0.539 0.627 b_cubed 0.630 0.444 0.521 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.866 0.598 0.708 strong_mention_match 0.841 0.581 0.687 strong_typed_mention_match 0.712 0.492 0.582 strong_typed_all_match 0.724 0.537 0.617 strong_typed_link_match 0.661 0.356 0.463 strong_typed_nil_match 0.786 0.636 0.703 entity_match 0.763 0.527 0.624 mention_ceaf 0.745 0.515 0.609 typed_mention_ceaf 0.688 0.475 0.562 typed_mention_ceaf_plus 0.753 0.430 0.547 b_cubed 0.674 0.379 0.485 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.896 0.714 0.795 strong_mention_match 0.857 0.683 0.760 strong_typed_mention_match 0.657 0.524 0.583 strong_typed_all_match 0.628 0.493 0.552 strong_typed_link_match 0.720 0.592 0.650 strong_typed_nil_match 0.752 0.584 0.657 entity_match 0.710 0.566 0.630 mention_ceaf 0.685 0.545 0.607 typed_mention_ceaf 0.617 0.491 0.547 typed_mention_ceaf_plus 0.730 0.482 0.581 b_cubed 0.607 0.408 0.488 b_cubed_plus ************************************************************* Run ID: UI_CCG5 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.863 0.692 0.768 strong_mention_match 0.831 0.666 0.739 strong_typed_mention_match 0.648 0.520 0.577 strong_typed_all_match 0.638 0.527 0.577 strong_typed_link_match 0.677 0.503 0.577 strong_typed_nil_match 0.762 0.625 0.686 entity_match 0.619 0.496 0.551 mention_ceaf 0.600 0.481 0.534 typed_mention_ceaf 0.548 0.439 0.487 typed_mention_ceaf_plus 0.554 0.453 0.498 b_cubed 0.469 0.385 0.423 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.885 0.763 0.820 strong_mention_match 0.851 0.733 0.788 strong_typed_mention_match 0.664 0.572 0.615 strong_typed_all_match 0.637 0.579 0.607 strong_typed_link_match 0.727 0.558 0.632 strong_typed_nil_match 0.745 0.677 0.710 entity_match 0.699 0.602 0.647 mention_ceaf 0.677 0.583 0.626 typed_mention_ceaf 0.597 0.514 0.553 typed_mention_ceaf_plus 0.730 0.542 0.623 b_cubed 0.614 0.446 0.517 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.827 0.619 0.708 strong_mention_match 0.803 0.601 0.687 strong_typed_mention_match 0.644 0.481 0.551 strong_typed_all_match 0.658 0.518 0.580 strong_typed_link_match 0.589 0.371 0.455 strong_typed_nil_match 0.787 0.616 0.691 entity_match 0.693 0.518 0.593 mention_ceaf 0.676 0.505 0.578 typed_mention_ceaf 0.621 0.464 0.531 typed_mention_ceaf_plus 0.650 0.427 0.515 b_cubed 0.569 0.366 0.445 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.888 0.728 0.800 strong_mention_match 0.846 0.694 0.763 strong_typed_mention_match 0.639 0.524 0.576 strong_typed_all_match 0.608 0.491 0.543 strong_typed_link_match 0.705 0.596 0.646 strong_typed_nil_match 0.750 0.590 0.661 entity_match 0.690 0.566 0.622 mention_ceaf 0.665 0.545 0.599 typed_mention_ceaf 0.599 0.491 0.540 typed_mention_ceaf_plus 0.703 0.488 0.576 b_cubed 0.579 0.407 0.478 b_cubed_plus