======================================================================== TAC KBP 2017 TRI-LINGUAL ENTITY DISCOVERY AND LINKING EVALUATION RESULTS ======================================================================== Team ID: BUPTTeam Organization: Beijing University of Posts and Telecommunications ************************************************************* Run ID: BUPTTeam1 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.874 0.611 0.719 strong_mention_match 0.808 0.565 0.665 strong_typed_mention_match 0.706 0.494 0.581 strong_typed_all_match 0.719 0.503 0.592 strong_typed_link_match 0.676 0.471 0.555 strong_typed_nil_match 0.675 0.609 0.640 entity_match 0.766 0.536 0.631 mention_ceaf 0.725 0.507 0.597 typed_mention_ceaf 0.669 0.468 0.551 typed_mention_ceaf_plus 0.773 0.447 0.566 b_cubed 0.657 0.400 0.497 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.894 0.631 0.739 strong_mention_match 0.834 0.588 0.690 strong_typed_mention_match 0.700 0.494 0.579 strong_typed_all_match 0.681 0.497 0.575 strong_typed_link_match 0.742 0.488 0.589 strong_typed_nil_match 0.663 0.624 0.643 entity_match 0.782 0.552 0.647 mention_ceaf 0.738 0.521 0.611 typed_mention_ceaf 0.655 0.462 0.542 typed_mention_ceaf_plus 0.798 0.482 0.601 b_cubed 0.646 0.417 0.507 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.917 0.620 0.740 strong_mention_match 0.874 0.591 0.705 strong_typed_mention_match 0.793 0.536 0.639 strong_typed_all_match 0.803 0.588 0.679 strong_typed_link_match 0.748 0.382 0.506 strong_typed_nil_match 0.789 0.674 0.727 entity_match 0.868 0.587 0.701 mention_ceaf 0.836 0.565 0.674 typed_mention_ceaf 0.780 0.527 0.629 typed_mention_ceaf_plus 0.844 0.496 0.625 b_cubed 0.770 0.453 0.570 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.797 0.579 0.671 strong_mention_match 0.698 0.507 0.587 strong_typed_mention_match 0.598 0.434 0.503 strong_typed_all_match 0.608 0.380 0.467 strong_typed_link_match 0.584 0.557 0.570 strong_typed_nil_match 0.571 0.525 0.547 entity_match 0.654 0.475 0.550 mention_ceaf 0.597 0.434 0.503 typed_mention_ceaf 0.536 0.390 0.451 typed_mention_ceaf_plus 0.681 0.402 0.506 b_cubed 0.537 0.350 0.424 b_cubed_plus ************************************************************* Run ID: BUPTTeam2 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.874 0.611 0.720 strong_mention_match 0.809 0.566 0.666 strong_typed_mention_match 0.707 0.495 0.582 strong_typed_all_match 0.720 0.504 0.593 strong_typed_link_match 0.678 0.471 0.556 strong_typed_nil_match 0.675 0.609 0.640 entity_match 0.768 0.537 0.632 mention_ceaf 0.727 0.508 0.598 typed_mention_ceaf 0.670 0.469 0.552 typed_mention_ceaf_plus 0.775 0.447 0.567 b_cubed 0.660 0.401 0.499 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.894 0.631 0.740 strong_mention_match 0.834 0.588 0.690 strong_typed_mention_match 0.700 0.494 0.580 strong_typed_all_match 0.681 0.497 0.575 strong_typed_link_match 0.743 0.489 0.590 strong_typed_nil_match 0.663 0.624 0.643 entity_match 0.782 0.552 0.647 mention_ceaf 0.738 0.521 0.611 typed_mention_ceaf 0.655 0.462 0.542 typed_mention_ceaf_plus 0.798 0.482 0.601 b_cubed 0.647 0.417 0.507 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.919 0.621 0.742 strong_mention_match 0.876 0.592 0.707 strong_typed_mention_match 0.795 0.538 0.641 strong_typed_all_match 0.804 0.590 0.681 strong_typed_link_match 0.755 0.382 0.507 strong_typed_nil_match 0.792 0.675 0.729 entity_match 0.871 0.589 0.703 mention_ceaf 0.838 0.567 0.676 typed_mention_ceaf 0.783 0.529 0.632 typed_mention_ceaf_plus 0.849 0.498 0.628 b_cubed 0.775 0.455 0.573 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.797 0.579 0.671 strong_mention_match 0.698 0.507 0.587 strong_typed_mention_match 0.598 0.434 0.503 strong_typed_all_match 0.608 0.380 0.467 strong_typed_link_match 0.584 0.557 0.570 strong_typed_nil_match 0.571 0.525 0.547 entity_match 0.654 0.475 0.550 mention_ceaf 0.597 0.434 0.503 typed_mention_ceaf 0.536 0.390 0.451 typed_mention_ceaf_plus 0.681 0.402 0.506 b_cubed 0.537 0.350 0.424 b_cubed_plus ************************************************************* Run ID: BUPTTeam3 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.876 0.615 0.723 strong_mention_match 0.812 0.570 0.669 strong_typed_mention_match 0.714 0.501 0.589 strong_typed_all_match 0.727 0.513 0.601 strong_typed_link_match 0.682 0.473 0.559 strong_typed_nil_match 0.681 0.611 0.644 entity_match 0.772 0.542 0.637 mention_ceaf 0.731 0.513 0.603 typed_mention_ceaf 0.677 0.475 0.558 typed_mention_ceaf_plus 0.776 0.454 0.573 b_cubed 0.667 0.408 0.506 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.894 0.636 0.743 strong_mention_match 0.834 0.594 0.694 strong_typed_mention_match 0.711 0.506 0.591 strong_typed_all_match 0.696 0.515 0.592 strong_typed_link_match 0.745 0.488 0.590 strong_typed_nil_match 0.670 0.626 0.647 entity_match 0.789 0.561 0.656 mention_ceaf 0.745 0.530 0.619 typed_mention_ceaf 0.666 0.474 0.554 typed_mention_ceaf_plus 0.795 0.494 0.609 b_cubed 0.654 0.430 0.519 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.922 0.625 0.745 strong_mention_match 0.880 0.596 0.711 strong_typed_mention_match 0.803 0.544 0.649 strong_typed_all_match 0.811 0.598 0.688 strong_typed_link_match 0.768 0.385 0.513 strong_typed_nil_match 0.803 0.680 0.737 entity_match 0.877 0.594 0.708 mention_ceaf 0.843 0.571 0.681 typed_mention_ceaf 0.791 0.536 0.639 typed_mention_ceaf_plus 0.851 0.504 0.633 b_cubed 0.786 0.464 0.584 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.800 0.581 0.673 strong_mention_match 0.700 0.509 0.589 strong_typed_mention_match 0.599 0.435 0.504 strong_typed_all_match 0.608 0.380 0.468 strong_typed_link_match 0.585 0.558 0.571 strong_typed_nil_match 0.571 0.525 0.547 entity_match 0.655 0.476 0.551 mention_ceaf 0.599 0.435 0.504 typed_mention_ceaf 0.537 0.390 0.452 typed_mention_ceaf_plus 0.685 0.403 0.508 b_cubed 0.537 0.351 0.424 b_cubed_plus ************************************************************* Run ID: BUPTTeam5 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.877 0.615 0.723 strong_mention_match 0.812 0.570 0.670 strong_typed_mention_match 0.715 0.501 0.589 strong_typed_all_match 0.727 0.513 0.602 strong_typed_link_match 0.684 0.473 0.560 strong_typed_nil_match 0.682 0.612 0.645 entity_match 0.773 0.542 0.637 mention_ceaf 0.732 0.514 0.604 typed_mention_ceaf 0.678 0.475 0.559 typed_mention_ceaf_plus 0.776 0.454 0.573 b_cubed 0.667 0.409 0.507 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.894 0.636 0.743 strong_mention_match 0.834 0.594 0.694 strong_typed_mention_match 0.712 0.506 0.592 strong_typed_all_match 0.697 0.516 0.593 strong_typed_link_match 0.745 0.488 0.590 strong_typed_nil_match 0.670 0.626 0.647 entity_match 0.789 0.561 0.656 mention_ceaf 0.745 0.530 0.620 typed_mention_ceaf 0.667 0.474 0.554 typed_mention_ceaf_plus 0.795 0.494 0.609 b_cubed 0.655 0.431 0.520 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.922 0.625 0.745 strong_mention_match 0.880 0.596 0.711 strong_typed_mention_match 0.804 0.544 0.649 strong_typed_all_match 0.811 0.598 0.688 strong_typed_link_match 0.769 0.385 0.513 strong_typed_nil_match 0.803 0.680 0.737 entity_match 0.877 0.594 0.708 mention_ceaf 0.843 0.571 0.681 typed_mention_ceaf 0.791 0.536 0.639 typed_mention_ceaf_plus 0.852 0.504 0.633 b_cubed 0.786 0.464 0.584 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.802 0.581 0.674 strong_mention_match 0.702 0.509 0.590 strong_typed_mention_match 0.600 0.435 0.504 strong_typed_all_match 0.607 0.380 0.467 strong_typed_link_match 0.589 0.558 0.573 strong_typed_nil_match 0.573 0.528 0.549 entity_match 0.658 0.477 0.553 mention_ceaf 0.601 0.436 0.505 typed_mention_ceaf 0.538 0.390 0.452 typed_mention_ceaf_plus 0.686 0.404 0.509 b_cubed 0.539 0.351 0.425 b_cubed_plus