======================================================================= TAC KBP 2015 TRILINGUAL ENTITY DISCOVERY AND LINKING EVALUATION RESULTS ======================================================================= Team ID: BUPTTeam Organization: Beijing University of Posts and Telecommunications ************************************************************* Run ID: BUPTTeam1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.796 0.727 0.760 strong_mention_match 0.756 0.691 0.722 strong_typed_mention_match 0.671 0.613 0.641 strong_typed_all_match 0.667 0.597 0.630 strong_typed_link_match 0.681 0.658 0.670 strong_typed_nil_match 0.627 0.642 0.635 entity_match 0.637 0.582 0.609 mention_ceaf 0.620 0.566 0.592 typed_mention_ceaf 0.590 0.539 0.563 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.831 0.675 0.745 strong_mention_match 0.796 0.647 0.714 strong_typed_mention_match 0.695 0.565 0.623 strong_typed_all_match 0.682 0.575 0.624 strong_typed_link_match 0.734 0.540 0.622 strong_typed_nil_match 0.650 0.656 0.653 entity_match 0.762 0.619 0.683 mention_ceaf 0.740 0.601 0.663 typed_mention_ceaf 0.688 0.559 0.616 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.818 0.777 0.797 strong_mention_match 0.788 0.748 0.767 strong_typed_mention_match 0.739 0.702 0.720 strong_typed_all_match 0.707 0.680 0.693 strong_typed_link_match 0.865 0.781 0.821 strong_typed_nil_match 0.677 0.675 0.676 entity_match 0.778 0.739 0.758 mention_ceaf 0.757 0.719 0.738 typed_mention_ceaf 0.724 0.688 0.706 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.692 0.768 0.728 strong_mention_match 0.629 0.698 0.662 strong_typed_mention_match 0.515 0.571 0.541 strong_typed_all_match 0.545 0.485 0.513 strong_typed_link_match 0.473 0.790 0.592 strong_typed_nil_match 0.529 0.575 0.551 entity_match 0.570 0.632 0.599 mention_ceaf 0.531 0.589 0.559 typed_mention_ceaf 0.471 0.522 0.495 typed_mention_ceaf_plus ************************************************************* Run ID: BUPTTeam2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.798 0.727 0.761 strong_mention_match 0.759 0.691 0.724 strong_typed_mention_match 0.674 0.613 0.642 strong_typed_all_match 0.671 0.597 0.632 strong_typed_link_match 0.681 0.658 0.670 strong_typed_nil_match 0.630 0.642 0.636 entity_match 0.640 0.582 0.610 mention_ceaf 0.622 0.566 0.593 typed_mention_ceaf 0.592 0.539 0.564 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.834 0.675 0.746 strong_mention_match 0.800 0.647 0.715 strong_typed_mention_match 0.698 0.565 0.625 strong_typed_all_match 0.686 0.575 0.625 strong_typed_link_match 0.734 0.540 0.622 strong_typed_nil_match 0.653 0.656 0.654 entity_match 0.765 0.619 0.684 mention_ceaf 0.743 0.601 0.664 typed_mention_ceaf 0.691 0.559 0.618 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.822 0.777 0.799 strong_mention_match 0.792 0.748 0.769 strong_typed_mention_match 0.744 0.702 0.722 strong_typed_all_match 0.712 0.680 0.696 strong_typed_link_match 0.865 0.781 0.821 strong_typed_nil_match 0.681 0.675 0.678 entity_match 0.782 0.739 0.760 mention_ceaf 0.761 0.719 0.740 typed_mention_ceaf 0.728 0.688 0.707 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.692 0.768 0.728 strong_mention_match 0.629 0.698 0.662 strong_typed_mention_match 0.515 0.571 0.541 strong_typed_all_match 0.545 0.485 0.513 strong_typed_link_match 0.473 0.790 0.592 strong_typed_nil_match 0.529 0.575 0.551 entity_match 0.570 0.632 0.599 mention_ceaf 0.531 0.589 0.559 typed_mention_ceaf 0.471 0.522 0.495 typed_mention_ceaf_plus ************************************************************* Run ID: BUPTTeam3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.796 0.717 0.755 strong_mention_match 0.758 0.683 0.719 strong_typed_mention_match 0.676 0.609 0.640 strong_typed_all_match 0.670 0.591 0.628 strong_typed_link_match 0.691 0.659 0.675 strong_typed_nil_match 0.632 0.647 0.639 entity_match 0.644 0.580 0.610 mention_ceaf 0.626 0.564 0.594 typed_mention_ceaf 0.595 0.536 0.564 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.835 0.654 0.734 strong_mention_match 0.801 0.627 0.703 strong_typed_mention_match 0.696 0.544 0.611 strong_typed_all_match 0.681 0.547 0.607 strong_typed_link_match 0.734 0.539 0.622 strong_typed_nil_match 0.653 0.646 0.650 entity_match 0.763 0.598 0.670 mention_ceaf 0.741 0.580 0.651 typed_mention_ceaf 0.686 0.537 0.603 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.821 0.777 0.798 strong_mention_match 0.790 0.748 0.769 strong_typed_mention_match 0.742 0.702 0.722 strong_typed_all_match 0.710 0.680 0.695 strong_typed_link_match 0.865 0.781 0.821 strong_typed_nil_match 0.678 0.675 0.677 entity_match 0.781 0.739 0.759 mention_ceaf 0.760 0.719 0.739 typed_mention_ceaf 0.727 0.688 0.707 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.687 0.771 0.726 strong_mention_match 0.630 0.707 0.666 strong_typed_mention_match 0.533 0.598 0.564 strong_typed_all_match 0.558 0.520 0.539 strong_typed_link_match 0.495 0.797 0.611 strong_typed_nil_match 0.544 0.611 0.575 entity_match 0.577 0.648 0.611 mention_ceaf 0.543 0.609 0.574 typed_mention_ceaf 0.490 0.550 0.519 typed_mention_ceaf_plus ************************************************************* Run ID: BUPTTeam4 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.797 0.727 0.761 strong_mention_match 0.759 0.693 0.724 strong_typed_mention_match 0.677 0.618 0.646 strong_typed_all_match 0.671 0.604 0.635 strong_typed_link_match 0.696 0.660 0.677 strong_typed_nil_match 0.632 0.651 0.641 entity_match 0.646 0.589 0.616 mention_ceaf 0.629 0.574 0.600 typed_mention_ceaf 0.598 0.546 0.571 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.831 0.675 0.745 strong_mention_match 0.796 0.647 0.714 strong_typed_mention_match 0.695 0.565 0.623 strong_typed_all_match 0.682 0.575 0.624 strong_typed_link_match 0.734 0.540 0.622 strong_typed_nil_match 0.650 0.656 0.653 entity_match 0.762 0.619 0.683 mention_ceaf 0.740 0.601 0.663 typed_mention_ceaf 0.688 0.559 0.616 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.821 0.777 0.798 strong_mention_match 0.790 0.748 0.769 strong_typed_mention_match 0.742 0.702 0.722 strong_typed_all_match 0.710 0.680 0.695 strong_typed_link_match 0.865 0.781 0.821 strong_typed_nil_match 0.678 0.675 0.677 entity_match 0.781 0.739 0.759 mention_ceaf 0.760 0.719 0.739 typed_mention_ceaf 0.727 0.688 0.707 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.694 0.771 0.731 strong_mention_match 0.637 0.707 0.670 strong_typed_mention_match 0.539 0.598 0.567 strong_typed_all_match 0.560 0.520 0.540 strong_typed_link_match 0.506 0.797 0.619 strong_typed_nil_match 0.547 0.611 0.577 entity_match 0.584 0.648 0.614 mention_ceaf 0.549 0.609 0.577 typed_mention_ceaf 0.496 0.550 0.522 typed_mention_ceaf_plus ************************************************************* Run ID: BUPTTeam5 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.817 0.631 0.712 strong_mention_match 0.774 0.598 0.675 strong_typed_mention_match 0.676 0.523 0.590 strong_typed_all_match 0.671 0.500 0.573 strong_typed_link_match 0.690 0.588 0.635 strong_typed_nil_match 0.648 0.549 0.594 entity_match 0.639 0.494 0.557 mention_ceaf 0.623 0.482 0.543 typed_mention_ceaf 0.589 0.455 0.514 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.863 0.566 0.683 strong_mention_match 0.818 0.536 0.648 strong_typed_mention_match 0.683 0.448 0.541 strong_typed_all_match 0.672 0.453 0.541 strong_typed_link_match 0.714 0.436 0.541 strong_typed_nil_match 0.656 0.525 0.583 entity_match 0.771 0.505 0.611 mention_ceaf 0.749 0.491 0.593 typed_mention_ceaf 0.676 0.443 0.535 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.825 0.718 0.768 strong_mention_match 0.794 0.690 0.738 strong_typed_mention_match 0.735 0.639 0.684 strong_typed_all_match 0.699 0.604 0.648 strong_typed_link_match 0.860 0.764 0.809 strong_typed_nil_match 0.679 0.632 0.655 entity_match 0.782 0.680 0.728 mention_ceaf 0.760 0.661 0.707 typed_mention_ceaf 0.718 0.625 0.668 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.715 0.640 0.676 strong_mention_match 0.654 0.585 0.618 strong_typed_mention_match 0.555 0.497 0.524 strong_typed_all_match 0.594 0.406 0.482 strong_typed_link_match 0.508 0.730 0.599 strong_typed_nil_match 0.585 0.486 0.531 entity_match 0.582 0.521 0.550 mention_ceaf 0.555 0.497 0.524 typed_mention_ceaf 0.503 0.450 0.475 typed_mention_ceaf_plus