================================================================================ TAC KBP 2016 ENTITY DISCOVERY AND LINKING EVALUATION RESULTS (SECOND EDL WINDOW) ================================================================================ Team ID: BUPTTeam Organization: Beijing University of Posts and Telecommunications ************************************************************* Run ID: BUPTTeam2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No Did the run use any of the distributed EDL1 runs (from the first evaluation window): No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.804 0.595 0.684 strong_typed_mention_match 0.700 0.518 0.595 strong_typed_all_match 0.744 0.495 0.595 strong_typed_link_match 0.597 0.597 0.597 strong_typed_nil_match 0.695 0.590 0.638 entity_match 0.749 0.555 0.637 mention_ceaf 0.717 0.530 0.610 typed_mention_ceaf 0.678 0.502 0.576 typed_mention_ceaf_plus 0.781 0.439 0.562 b_cubed 0.663 0.407 0.504 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.913 0.635 0.749 strong_mention_match 0.866 0.602 0.710 strong_typed_mention_match 0.699 0.486 0.573 strong_typed_all_match 0.742 0.457 0.566 strong_typed_link_match 0.602 0.587 0.594 strong_typed_nil_match 0.728 0.589 0.651 entity_match 0.767 0.533 0.629 mention_ceaf 0.736 0.512 0.604 typed_mention_ceaf 0.674 0.469 0.553 typed_mention_ceaf_plus 0.849 0.425 0.566 b_cubed 0.675 0.378 0.485 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.850 0.639 0.730 strong_mention_match 0.810 0.609 0.695 strong_typed_mention_match 0.762 0.573 0.654 strong_typed_all_match 0.803 0.571 0.668 strong_typed_link_match 0.632 0.579 0.604 strong_typed_nil_match 0.761 0.631 0.690 entity_match 0.821 0.617 0.704 mention_ceaf 0.784 0.589 0.673 typed_mention_ceaf 0.752 0.565 0.645 typed_mention_ceaf_plus 0.782 0.517 0.623 b_cubed 0.728 0.489 0.585 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.798 0.627 0.702 strong_mention_match 0.725 0.569 0.638 strong_typed_mention_match 0.625 0.491 0.550 strong_typed_all_match 0.661 0.443 0.531 strong_typed_link_match 0.564 0.626 0.593 strong_typed_nil_match 0.599 0.549 0.573 entity_match 0.694 0.545 0.611 mention_ceaf 0.654 0.513 0.575 typed_mention_ceaf 0.592 0.465 0.521 typed_mention_ceaf_plus 0.717 0.448 0.552 b_cubed 0.580 0.393 0.468 b_cubed_plus ************************************************************* Run ID: BUPTTeam3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No Did the run use any of the distributed EDL1 runs (from the first evaluation window): No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.807 0.581 0.676 strong_typed_mention_match 0.706 0.508 0.591 strong_typed_all_match 0.743 0.485 0.587 strong_typed_link_match 0.616 0.590 0.603 strong_typed_nil_match 0.693 0.580 0.631 entity_match 0.759 0.546 0.635 mention_ceaf 0.726 0.523 0.608 typed_mention_ceaf 0.684 0.492 0.573 typed_mention_ceaf_plus 0.787 0.429 0.555 b_cubed 0.670 0.396 0.498 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.919 0.614 0.736 strong_mention_match 0.874 0.583 0.700 strong_typed_mention_match 0.721 0.482 0.578 strong_typed_all_match 0.742 0.454 0.563 strong_typed_link_match 0.670 0.583 0.623 strong_typed_nil_match 0.726 0.586 0.648 entity_match 0.790 0.528 0.633 mention_ceaf 0.759 0.507 0.608 typed_mention_ceaf 0.696 0.465 0.557 typed_mention_ceaf_plus 0.858 0.419 0.563 b_cubed 0.697 0.374 0.487 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.854 0.617 0.717 strong_mention_match 0.813 0.588 0.682 strong_typed_mention_match 0.759 0.549 0.637 strong_typed_all_match 0.802 0.546 0.650 strong_typed_link_match 0.624 0.561 0.591 strong_typed_nil_match 0.759 0.602 0.671 entity_match 0.827 0.597 0.694 mention_ceaf 0.790 0.571 0.663 typed_mention_ceaf 0.751 0.543 0.631 typed_mention_ceaf_plus 0.792 0.492 0.607 b_cubed 0.729 0.462 0.566 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.798 0.627 0.702 strong_mention_match 0.725 0.569 0.638 strong_typed_mention_match 0.625 0.491 0.550 strong_typed_all_match 0.661 0.443 0.531 strong_typed_link_match 0.564 0.626 0.593 strong_typed_nil_match 0.599 0.549 0.573 entity_match 0.694 0.545 0.611 mention_ceaf 0.654 0.513 0.575 typed_mention_ceaf 0.592 0.465 0.521 typed_mention_ceaf_plus 0.717 0.448 0.552 b_cubed 0.580 0.393 0.468 b_cubed_plus ************************************************************* Run ID: BUPTTeam4 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No Did the run use any of the distributed EDL1 runs (from the first evaluation window): No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.806 0.589 0.680 strong_typed_mention_match 0.708 0.517 0.597 strong_typed_all_match 0.745 0.494 0.594 strong_typed_link_match 0.619 0.596 0.607 strong_typed_nil_match 0.695 0.590 0.638 entity_match 0.757 0.553 0.639 mention_ceaf 0.724 0.529 0.612 typed_mention_ceaf 0.685 0.500 0.578 typed_mention_ceaf_plus 0.783 0.437 0.561 b_cubed 0.670 0.405 0.505 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.918 0.614 0.736 strong_mention_match 0.873 0.584 0.700 strong_typed_mention_match 0.722 0.483 0.578 strong_typed_all_match 0.742 0.454 0.564 strong_typed_link_match 0.671 0.583 0.624 strong_typed_nil_match 0.727 0.589 0.651 entity_match 0.790 0.528 0.633 mention_ceaf 0.759 0.507 0.608 typed_mention_ceaf 0.696 0.465 0.558 typed_mention_ceaf_plus 0.858 0.419 0.563 b_cubed 0.697 0.374 0.487 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.850 0.639 0.730 strong_mention_match 0.810 0.609 0.695 strong_typed_mention_match 0.762 0.573 0.654 strong_typed_all_match 0.803 0.571 0.668 strong_typed_link_match 0.632 0.579 0.604 strong_typed_nil_match 0.761 0.631 0.690 entity_match 0.821 0.617 0.704 mention_ceaf 0.784 0.589 0.673 typed_mention_ceaf 0.752 0.565 0.645 typed_mention_ceaf_plus 0.782 0.517 0.623 b_cubed 0.728 0.489 0.585 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.798 0.627 0.702 strong_mention_match 0.725 0.569 0.638 strong_typed_mention_match 0.625 0.491 0.550 strong_typed_all_match 0.661 0.443 0.531 strong_typed_link_match 0.564 0.626 0.593 strong_typed_nil_match 0.599 0.549 0.573 entity_match 0.694 0.545 0.611 mention_ceaf 0.654 0.513 0.575 typed_mention_ceaf 0.592 0.465 0.521 typed_mention_ceaf_plus 0.717 0.448 0.552 b_cubed 0.580 0.393 0.468 b_cubed_plus ************************************************************* Run ID: BUPTTeam5 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No Did the run use any of the distributed EDL1 runs (from the first evaluation window): No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.807 0.589 0.680 strong_typed_mention_match 0.708 0.517 0.598 strong_typed_all_match 0.745 0.494 0.594 strong_typed_link_match 0.620 0.596 0.607 strong_typed_nil_match 0.695 0.590 0.638 entity_match 0.758 0.553 0.639 mention_ceaf 0.725 0.529 0.612 typed_mention_ceaf 0.686 0.500 0.579 typed_mention_ceaf_plus 0.784 0.437 0.561 b_cubed 0.671 0.405 0.505 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.919 0.614 0.736 strong_mention_match 0.875 0.584 0.700 strong_typed_mention_match 0.723 0.483 0.579 strong_typed_all_match 0.743 0.454 0.564 strong_typed_link_match 0.673 0.583 0.625 strong_typed_nil_match 0.727 0.589 0.651 entity_match 0.791 0.528 0.633 mention_ceaf 0.760 0.507 0.609 typed_mention_ceaf 0.697 0.465 0.558 typed_mention_ceaf_plus 0.859 0.419 0.563 b_cubed 0.698 0.374 0.487 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.850 0.639 0.730 strong_mention_match 0.810 0.609 0.695 strong_typed_mention_match 0.762 0.573 0.654 strong_typed_all_match 0.803 0.571 0.668 strong_typed_link_match 0.632 0.579 0.604 strong_typed_nil_match 0.762 0.631 0.690 entity_match 0.821 0.617 0.704 mention_ceaf 0.784 0.589 0.673 typed_mention_ceaf 0.752 0.565 0.645 typed_mention_ceaf_plus 0.783 0.517 0.623 b_cubed 0.728 0.489 0.585 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.799 0.627 0.703 strong_mention_match 0.726 0.569 0.638 strong_typed_mention_match 0.626 0.491 0.550 strong_typed_all_match 0.662 0.443 0.531 strong_typed_link_match 0.564 0.626 0.593 strong_typed_nil_match 0.600 0.548 0.573 entity_match 0.695 0.545 0.611 mention_ceaf 0.655 0.513 0.576 typed_mention_ceaf 0.593 0.465 0.521 typed_mention_ceaf_plus 0.719 0.448 0.552 b_cubed 0.582 0.393 0.469 b_cubed_plus