======================================================================== TAC KBP 2017 TRI-LINGUAL ENTITY DISCOVERY AND LINKING EVALUATION RESULTS ======================================================================== Team ID: Ugglan Organization: Lund University ************************************************************* Run ID: Ugglan1 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.867 0.615 0.719 strong_mention_match 0.815 0.578 0.676 strong_typed_mention_match 0.707 0.501 0.587 strong_typed_all_match 0.776 0.519 0.622 strong_typed_link_match 0.569 0.458 0.507 strong_typed_nil_match 0.806 0.628 0.706 entity_match 0.752 0.533 0.624 mention_ceaf 0.718 0.509 0.595 typed_mention_ceaf 0.671 0.476 0.557 typed_mention_ceaf_plus 0.755 0.449 0.563 b_cubed 0.649 0.410 0.503 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.883 0.629 0.734 strong_mention_match 0.841 0.598 0.699 strong_typed_mention_match 0.736 0.524 0.612 strong_typed_all_match 0.735 0.557 0.634 strong_typed_link_match 0.736 0.459 0.565 strong_typed_nil_match 0.764 0.652 0.704 entity_match 0.778 0.554 0.647 mention_ceaf 0.748 0.533 0.622 typed_mention_ceaf 0.693 0.493 0.576 typed_mention_ceaf_plus 0.770 0.496 0.604 b_cubed 0.666 0.452 0.538 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.845 0.622 0.717 strong_mention_match 0.778 0.573 0.660 strong_typed_mention_match 0.651 0.479 0.552 strong_typed_all_match 0.790 0.504 0.616 strong_typed_link_match 0.394 0.405 0.399 strong_typed_nil_match 0.815 0.641 0.718 entity_match 0.757 0.557 0.642 mention_ceaf 0.712 0.524 0.604 typed_mention_ceaf 0.640 0.471 0.543 typed_mention_ceaf_plus 0.769 0.449 0.567 b_cubed 0.638 0.391 0.484 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.885 0.591 0.708 strong_mention_match 0.848 0.566 0.679 strong_typed_mention_match 0.766 0.511 0.613 strong_typed_all_match 0.801 0.507 0.621 strong_typed_link_match 0.699 0.520 0.596 strong_typed_nil_match 0.839 0.595 0.696 entity_match 0.765 0.511 0.612 mention_ceaf 0.740 0.494 0.592 typed_mention_ceaf 0.700 0.467 0.560 typed_mention_ceaf_plus 0.747 0.461 0.570 b_cubed 0.669 0.432 0.525 b_cubed_plus ************************************************************* Run ID: Ugglan2 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.827 0.673 0.742 strong_mention_match 0.782 0.636 0.701 strong_typed_mention_match 0.645 0.525 0.579 strong_typed_all_match 0.789 0.506 0.616 strong_typed_link_match 0.465 0.572 0.513 strong_typed_nil_match 0.802 0.633 0.707 entity_match 0.671 0.546 0.602 mention_ceaf 0.642 0.523 0.577 typed_mention_ceaf 0.599 0.488 0.538 typed_mention_ceaf_plus 0.734 0.457 0.563 b_cubed 0.598 0.415 0.490 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.844 0.748 0.793 strong_mention_match 0.802 0.711 0.754 strong_typed_mention_match 0.646 0.573 0.608 strong_typed_all_match 0.788 0.530 0.634 strong_typed_link_match 0.504 0.658 0.571 strong_typed_nil_match 0.775 0.657 0.711 entity_match 0.663 0.588 0.623 mention_ceaf 0.636 0.564 0.598 typed_mention_ceaf 0.585 0.518 0.549 typed_mention_ceaf_plus 0.772 0.523 0.623 b_cubed 0.608 0.467 0.528 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.827 0.598 0.694 strong_mention_match 0.775 0.560 0.650 strong_typed_mention_match 0.657 0.475 0.551 strong_typed_all_match 0.774 0.493 0.602 strong_typed_link_match 0.429 0.420 0.425 strong_typed_nil_match 0.797 0.647 0.714 entity_match 0.733 0.530 0.615 mention_ceaf 0.697 0.504 0.585 typed_mention_ceaf 0.645 0.466 0.541 typed_mention_ceaf_plus 0.761 0.416 0.538 b_cubed 0.644 0.372 0.471 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.811 0.708 0.756 strong_mention_match 0.769 0.672 0.717 strong_typed_mention_match 0.631 0.551 0.589 strong_typed_all_match 0.812 0.504 0.622 strong_typed_link_match 0.457 0.657 0.539 strong_typed_nil_match 0.835 0.598 0.696 entity_match 0.639 0.559 0.596 mention_ceaf 0.615 0.538 0.574 typed_mention_ceaf 0.565 0.494 0.527 typed_mention_ceaf_plus 0.708 0.503 0.588 b_cubed 0.565 0.458 0.506 b_cubed_plus ************************************************************* Run ID: Ugglan3 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.860 0.631 0.728 strong_mention_match 0.804 0.590 0.680 strong_typed_mention_match 0.682 0.501 0.578 strong_typed_all_match 0.786 0.507 0.616 strong_typed_link_match 0.512 0.486 0.499 strong_typed_nil_match 0.796 0.625 0.700 entity_match 0.733 0.538 0.620 mention_ceaf 0.697 0.512 0.590 typed_mention_ceaf 0.645 0.473 0.546 typed_mention_ceaf_plus 0.749 0.452 0.564 b_cubed 0.627 0.407 0.494 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.874 0.658 0.751 strong_mention_match 0.824 0.620 0.707 strong_typed_mention_match 0.683 0.514 0.587 strong_typed_all_match 0.821 0.504 0.624 strong_typed_link_match 0.522 0.534 0.528 strong_typed_nil_match 0.832 0.610 0.704 entity_match 0.753 0.566 0.646 mention_ceaf 0.719 0.541 0.618 typed_mention_ceaf 0.635 0.478 0.545 typed_mention_ceaf_plus 0.784 0.500 0.611 b_cubed 0.631 0.433 0.513 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.845 0.622 0.717 strong_mention_match 0.778 0.573 0.660 strong_typed_mention_match 0.651 0.479 0.552 strong_typed_all_match 0.790 0.504 0.616 strong_typed_link_match 0.394 0.405 0.399 strong_typed_nil_match 0.815 0.641 0.718 entity_match 0.757 0.557 0.642 mention_ceaf 0.712 0.524 0.604 typed_mention_ceaf 0.640 0.471 0.543 typed_mention_ceaf_plus 0.769 0.449 0.567 b_cubed 0.638 0.391 0.484 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.869 0.619 0.723 strong_mention_match 0.821 0.585 0.683 strong_typed_mention_match 0.728 0.519 0.606 strong_typed_all_match 0.753 0.514 0.611 strong_typed_link_match 0.678 0.530 0.595 strong_typed_nil_match 0.750 0.620 0.679 entity_match 0.730 0.520 0.607 mention_ceaf 0.701 0.500 0.583 typed_mention_ceaf 0.664 0.473 0.553 typed_mention_ceaf_plus 0.732 0.471 0.573 b_cubed 0.640 0.440 0.521 b_cubed_plus ************************************************************* Run ID: Ugglan4 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.894 0.584 0.706 strong_mention_match 0.830 0.543 0.656 strong_typed_mention_match 0.749 0.489 0.592 strong_typed_all_match 0.794 0.500 0.614 strong_typed_link_match 0.653 0.463 0.542 strong_typed_nil_match 0.801 0.617 0.697 entity_match 0.791 0.517 0.625 mention_ceaf 0.751 0.491 0.593 typed_mention_ceaf 0.709 0.464 0.561 typed_mention_ceaf_plus 0.777 0.429 0.553 b_cubed 0.688 0.395 0.502 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.906 0.650 0.757 strong_mention_match 0.838 0.601 0.700 strong_typed_mention_match 0.743 0.533 0.621 strong_typed_all_match 0.803 0.538 0.644 strong_typed_link_match 0.647 0.524 0.579 strong_typed_nil_match 0.823 0.621 0.707 entity_match 0.805 0.577 0.673 mention_ceaf 0.757 0.543 0.632 typed_mention_ceaf 0.694 0.498 0.580 typed_mention_ceaf_plus 0.801 0.512 0.625 b_cubed 0.682 0.457 0.547 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.891 0.574 0.698 strong_mention_match 0.824 0.531 0.646 strong_typed_mention_match 0.723 0.465 0.566 strong_typed_all_match 0.826 0.493 0.617 strong_typed_link_match 0.489 0.383 0.430 strong_typed_nil_match 0.854 0.643 0.734 entity_match 0.823 0.530 0.645 mention_ceaf 0.776 0.500 0.608 typed_mention_ceaf 0.712 0.458 0.558 typed_mention_ceaf_plus 0.818 0.416 0.552 b_cubed 0.713 0.371 0.488 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.883 0.536 0.667 strong_mention_match 0.831 0.504 0.628 strong_typed_mention_match 0.794 0.482 0.600 strong_typed_all_match 0.740 0.477 0.580 strong_typed_link_match 0.944 0.492 0.647 strong_typed_nil_match 0.729 0.585 0.649 entity_match 0.764 0.464 0.577 mention_ceaf 0.737 0.447 0.556 typed_mention_ceaf 0.723 0.439 0.546 typed_mention_ceaf_plus 0.718 0.421 0.531 b_cubed 0.681 0.406 0.509 b_cubed_plus ************************************************************* Run ID: Ugglan5 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.866 0.610 0.716 strong_mention_match 0.795 0.559 0.657 strong_typed_mention_match 0.700 0.492 0.578 strong_typed_all_match 0.701 0.507 0.588 strong_typed_link_match 0.698 0.457 0.552 strong_typed_nil_match 0.702 0.637 0.668 entity_match 0.745 0.524 0.616 mention_ceaf 0.707 0.497 0.584 typed_mention_ceaf 0.664 0.467 0.548 typed_mention_ceaf_plus 0.721 0.438 0.545 b_cubed 0.627 0.395 0.485 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.873 0.665 0.755 strong_mention_match 0.798 0.608 0.690 strong_typed_mention_match 0.693 0.529 0.600 strong_typed_all_match 0.662 0.547 0.599 strong_typed_link_match 0.773 0.493 0.602 strong_typed_nil_match 0.668 0.647 0.658 entity_match 0.751 0.573 0.650 mention_ceaf 0.705 0.538 0.610 typed_mention_ceaf 0.649 0.495 0.562 typed_mention_ceaf_plus 0.740 0.519 0.610 b_cubed 0.620 0.449 0.521 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.878 0.572 0.693 strong_mention_match 0.809 0.528 0.639 strong_typed_mention_match 0.698 0.455 0.551 strong_typed_all_match 0.777 0.479 0.593 strong_typed_link_match 0.505 0.383 0.435 strong_typed_nil_match 0.824 0.633 0.716 entity_match 0.799 0.521 0.631 mention_ceaf 0.752 0.491 0.594 typed_mention_ceaf 0.687 0.448 0.542 typed_mention_ceaf_plus 0.788 0.409 0.538 b_cubed 0.678 0.359 0.469 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.845 0.609 0.708 strong_mention_match 0.775 0.558 0.649 strong_typed_mention_match 0.709 0.511 0.594 strong_typed_all_match 0.647 0.513 0.573 strong_typed_link_match 0.908 0.504 0.649 strong_typed_nil_match 0.629 0.631 0.630 entity_match 0.712 0.513 0.597 mention_ceaf 0.679 0.489 0.569 typed_mention_ceaf 0.649 0.468 0.544 typed_mention_ceaf_plus 0.671 0.466 0.550 b_cubed 0.603 0.432 0.504 b_cubed_plus