=============================================================================== TAC KBP 2016 ENTITY DISCOVERY AND LINKING EVALUATION RESULTS (FIRST EDL WINDOW) =============================================================================== Team ID: BUPTTeam Organization: Beijing University of Posts and Telecommunications ************************************************************* Run ID: BUPTTeam1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.807 0.572 0.669 strong_typed_mention_match 0.712 0.505 0.590 strong_typed_all_match 0.744 0.484 0.586 strong_typed_link_match 0.631 0.577 0.603 strong_typed_nil_match 0.694 0.579 0.631 entity_match 0.767 0.544 0.637 mention_ceaf 0.734 0.520 0.609 typed_mention_ceaf 0.691 0.490 0.574 typed_mention_ceaf_plus 0.787 0.426 0.553 b_cubed 0.676 0.394 0.498 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.922 0.588 0.718 strong_mention_match 0.875 0.559 0.682 strong_typed_mention_match 0.740 0.472 0.577 strong_typed_all_match 0.744 0.452 0.562 strong_typed_link_match 0.728 0.547 0.625 strong_typed_nil_match 0.729 0.585 0.649 entity_match 0.817 0.521 0.636 mention_ceaf 0.784 0.501 0.611 typed_mention_ceaf 0.718 0.458 0.559 typed_mention_ceaf_plus 0.862 0.410 0.555 b_cubed 0.716 0.367 0.485 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.854 0.617 0.717 strong_mention_match 0.813 0.588 0.682 strong_typed_mention_match 0.759 0.549 0.637 strong_typed_all_match 0.802 0.546 0.650 strong_typed_link_match 0.624 0.561 0.591 strong_typed_nil_match 0.759 0.602 0.671 entity_match 0.827 0.597 0.694 mention_ceaf 0.790 0.571 0.663 typed_mention_ceaf 0.751 0.543 0.631 typed_mention_ceaf_plus 0.792 0.492 0.607 b_cubed 0.729 0.462 0.566 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.798 0.627 0.702 strong_mention_match 0.725 0.569 0.638 strong_typed_mention_match 0.625 0.491 0.550 strong_typed_all_match 0.661 0.443 0.531 strong_typed_link_match 0.564 0.626 0.593 strong_typed_nil_match 0.599 0.549 0.573 entity_match 0.694 0.545 0.611 mention_ceaf 0.654 0.513 0.575 typed_mention_ceaf 0.592 0.465 0.521 typed_mention_ceaf_plus 0.717 0.448 0.552 b_cubed 0.580 0.393 0.468 b_cubed_plus ************************************************************* Run ID: BUPTTeam3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.807 0.581 0.676 strong_typed_mention_match 0.706 0.508 0.591 strong_typed_all_match 0.743 0.485 0.587 strong_typed_link_match 0.616 0.590 0.603 strong_typed_nil_match 0.693 0.580 0.631 entity_match 0.759 0.546 0.635 mention_ceaf 0.726 0.523 0.608 typed_mention_ceaf 0.684 0.492 0.573 typed_mention_ceaf_plus 0.787 0.429 0.555 b_cubed 0.670 0.396 0.498 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.919 0.614 0.736 strong_mention_match 0.874 0.583 0.700 strong_typed_mention_match 0.721 0.482 0.578 strong_typed_all_match 0.742 0.454 0.563 strong_typed_link_match 0.670 0.583 0.623 strong_typed_nil_match 0.726 0.586 0.648 entity_match 0.790 0.528 0.633 mention_ceaf 0.759 0.507 0.608 typed_mention_ceaf 0.696 0.465 0.557 typed_mention_ceaf_plus 0.858 0.419 0.563 b_cubed 0.697 0.374 0.487 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.854 0.617 0.717 strong_mention_match 0.813 0.588 0.682 strong_typed_mention_match 0.759 0.549 0.637 strong_typed_all_match 0.802 0.546 0.650 strong_typed_link_match 0.624 0.561 0.591 strong_typed_nil_match 0.759 0.602 0.671 entity_match 0.827 0.597 0.694 mention_ceaf 0.790 0.571 0.663 typed_mention_ceaf 0.751 0.543 0.631 typed_mention_ceaf_plus 0.792 0.492 0.607 b_cubed 0.729 0.462 0.566 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.798 0.627 0.702 strong_mention_match 0.725 0.569 0.638 strong_typed_mention_match 0.625 0.491 0.550 strong_typed_all_match 0.661 0.443 0.531 strong_typed_link_match 0.564 0.626 0.593 strong_typed_nil_match 0.599 0.549 0.573 entity_match 0.694 0.545 0.611 mention_ceaf 0.654 0.513 0.575 typed_mention_ceaf 0.592 0.465 0.521 typed_mention_ceaf_plus 0.717 0.448 0.552 b_cubed 0.580 0.393 0.468 b_cubed_plus ************************************************************* Run ID: BUPTTeam5 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.805 0.570 0.668 strong_typed_mention_match 0.710 0.503 0.589 strong_typed_all_match 0.744 0.484 0.586 strong_typed_link_match 0.626 0.569 0.596 strong_typed_nil_match 0.694 0.579 0.631 entity_match 0.766 0.542 0.635 mention_ceaf 0.733 0.519 0.607 typed_mention_ceaf 0.690 0.488 0.572 typed_mention_ceaf_plus 0.786 0.424 0.551 b_cubed 0.674 0.392 0.496 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.921 0.586 0.716 strong_mention_match 0.874 0.556 0.680 strong_typed_mention_match 0.738 0.470 0.574 strong_typed_all_match 0.744 0.452 0.562 strong_typed_link_match 0.721 0.536 0.615 strong_typed_nil_match 0.729 0.585 0.649 entity_match 0.815 0.519 0.634 mention_ceaf 0.783 0.498 0.609 typed_mention_ceaf 0.716 0.456 0.557 typed_mention_ceaf_plus 0.861 0.407 0.553 b_cubed 0.715 0.364 0.483 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.852 0.615 0.714 strong_mention_match 0.811 0.585 0.680 strong_typed_mention_match 0.757 0.546 0.635 strong_typed_all_match 0.802 0.546 0.650 strong_typed_link_match 0.615 0.549 0.580 strong_typed_nil_match 0.759 0.602 0.671 entity_match 0.825 0.595 0.691 mention_ceaf 0.788 0.568 0.660 typed_mention_ceaf 0.749 0.541 0.628 typed_mention_ceaf_plus 0.790 0.490 0.605 b_cubed 0.727 0.460 0.563 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.798 0.627 0.702 strong_mention_match 0.725 0.569 0.638 strong_typed_mention_match 0.625 0.491 0.550 strong_typed_all_match 0.661 0.443 0.531 strong_typed_link_match 0.564 0.626 0.593 strong_typed_nil_match 0.599 0.549 0.573 entity_match 0.694 0.545 0.611 mention_ceaf 0.654 0.513 0.575 typed_mention_ceaf 0.592 0.465 0.521 typed_mention_ceaf_plus 0.717 0.448 0.552 b_cubed 0.580 0.393 0.468 b_cubed_plus