=============================================================================== TAC KBP 2016 ENTITY DISCOVERY AND LINKING EVALUATION RESULTS (FIRST EDL WINDOW) =============================================================================== Team ID: USTC_NELSLIP Organization: University of Science and Technology of China ************************************************************* Run ID: USTC_NELSLIP1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.822 0.704 0.759 strong_typed_mention_match 0.701 0.600 0.647 strong_typed_all_match 0.772 0.586 0.666 strong_typed_link_match 0.544 0.652 0.593 strong_typed_nil_match 0.823 0.678 0.743 entity_match 0.740 0.633 0.682 mention_ceaf 0.716 0.613 0.660 typed_mention_ceaf 0.676 0.579 0.624 typed_mention_ceaf_plus 0.726 0.546 0.624 b_cubed 0.626 0.501 0.557 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.895 0.751 0.817 strong_mention_match 0.846 0.710 0.772 strong_typed_mention_match 0.731 0.613 0.666 strong_typed_all_match 0.765 0.608 0.678 strong_typed_link_match 0.633 0.628 0.630 strong_typed_nil_match 0.836 0.695 0.759 entity_match 0.764 0.640 0.697 mention_ceaf 0.741 0.622 0.676 typed_mention_ceaf 0.703 0.590 0.642 typed_mention_ceaf_plus 0.741 0.569 0.644 b_cubed 0.653 0.519 0.579 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.837 0.781 0.808 strong_mention_match 0.789 0.737 0.762 strong_typed_mention_match 0.673 0.628 0.650 strong_typed_all_match 0.771 0.613 0.683 strong_typed_link_match 0.459 0.688 0.551 strong_typed_nil_match 0.825 0.671 0.740 entity_match 0.752 0.701 0.726 mention_ceaf 0.727 0.678 0.702 typed_mention_ceaf 0.658 0.614 0.636 typed_mention_ceaf_plus 0.692 0.638 0.664 b_cubed 0.594 0.562 0.577 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.881 0.690 0.774 strong_mention_match 0.839 0.656 0.736 strong_typed_mention_match 0.701 0.549 0.616 strong_typed_all_match 0.787 0.515 0.623 strong_typed_link_match 0.563 0.644 0.601 strong_typed_nil_match 0.804 0.664 0.728 entity_match 0.753 0.590 0.662 mention_ceaf 0.723 0.566 0.635 typed_mention_ceaf 0.666 0.521 0.585 typed_mention_ceaf_plus 0.785 0.506 0.615 b_cubed 0.654 0.455 0.536 b_cubed_plus ************************************************************* Run ID: USTC_NELSLIP2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.850 0.678 0.754 strong_typed_mention_match 0.730 0.582 0.647 strong_typed_all_match 0.795 0.565 0.661 strong_typed_link_match 0.582 0.639 0.609 strong_typed_nil_match 0.839 0.651 0.733 entity_match 0.767 0.611 0.680 mention_ceaf 0.745 0.594 0.661 typed_mention_ceaf 0.704 0.561 0.625 typed_mention_ceaf_plus 0.759 0.519 0.616 b_cubed 0.658 0.478 0.554 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.909 0.710 0.797 strong_mention_match 0.864 0.675 0.758 strong_typed_mention_match 0.754 0.589 0.661 strong_typed_all_match 0.787 0.583 0.670 strong_typed_link_match 0.660 0.610 0.634 strong_typed_nil_match 0.850 0.662 0.744 entity_match 0.787 0.614 0.690 mention_ceaf 0.767 0.598 0.672 typed_mention_ceaf 0.728 0.568 0.638 typed_mention_ceaf_plus 0.766 0.535 0.630 b_cubed 0.680 0.492 0.571 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.870 0.756 0.809 strong_mention_match 0.825 0.717 0.767 strong_typed_mention_match 0.708 0.615 0.658 strong_typed_all_match 0.796 0.600 0.684 strong_typed_link_match 0.503 0.675 0.576 strong_typed_nil_match 0.846 0.643 0.730 entity_match 0.786 0.683 0.731 mention_ceaf 0.763 0.663 0.710 typed_mention_ceaf 0.692 0.601 0.643 typed_mention_ceaf_plus 0.731 0.616 0.668 b_cubed 0.631 0.545 0.585 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.909 0.662 0.766 strong_mention_match 0.869 0.632 0.732 strong_typed_mention_match 0.728 0.530 0.613 strong_typed_all_match 0.806 0.491 0.610 strong_typed_link_match 0.602 0.639 0.620 strong_typed_nil_match 0.818 0.645 0.721 entity_match 0.778 0.566 0.656 mention_ceaf 0.750 0.546 0.631 typed_mention_ceaf 0.689 0.502 0.581 typed_mention_ceaf_plus 0.817 0.477 0.603 b_cubed 0.682 0.430 0.527 b_cubed_plus ************************************************************* Run ID: USTC_NELSLIP3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.836 0.681 0.751 strong_typed_mention_match 0.718 0.585 0.644 strong_typed_all_match 0.787 0.569 0.660 strong_typed_link_match 0.564 0.641 0.600 strong_typed_nil_match 0.838 0.660 0.738 entity_match 0.754 0.614 0.677 mention_ceaf 0.731 0.595 0.656 typed_mention_ceaf 0.693 0.565 0.622 typed_mention_ceaf_plus 0.743 0.521 0.612 b_cubed 0.644 0.480 0.550 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.907 0.745 0.818 strong_mention_match 0.857 0.704 0.773 strong_typed_mention_match 0.742 0.609 0.669 strong_typed_all_match 0.774 0.605 0.679 strong_typed_link_match 0.649 0.626 0.637 strong_typed_nil_match 0.847 0.690 0.760 entity_match 0.775 0.637 0.699 mention_ceaf 0.752 0.618 0.679 typed_mention_ceaf 0.715 0.587 0.645 typed_mention_ceaf_plus 0.754 0.564 0.645 b_cubed 0.665 0.516 0.581 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.846 0.730 0.783 strong_mention_match 0.802 0.692 0.743 strong_typed_mention_match 0.693 0.598 0.642 strong_typed_all_match 0.790 0.583 0.671 strong_typed_link_match 0.478 0.658 0.554 strong_typed_nil_match 0.850 0.640 0.730 entity_match 0.768 0.663 0.711 mention_ceaf 0.745 0.643 0.690 typed_mention_ceaf 0.679 0.586 0.629 typed_mention_ceaf_plus 0.711 0.588 0.644 b_cubed 0.617 0.524 0.566 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.897 0.668 0.766 strong_mention_match 0.855 0.637 0.730 strong_typed_mention_match 0.718 0.535 0.614 strong_typed_all_match 0.803 0.498 0.615 strong_typed_link_match 0.584 0.641 0.611 strong_typed_nil_match 0.817 0.643 0.720 entity_match 0.768 0.573 0.656 mention_ceaf 0.740 0.551 0.632 typed_mention_ceaf 0.681 0.508 0.582 typed_mention_ceaf_plus 0.801 0.485 0.605 b_cubed 0.671 0.438 0.530 b_cubed_plus