======================================================================== TAC KBP 2017 TRI-LINGUAL ENTITY DISCOVERY AND LINKING EVALUATION RESULTS ======================================================================== Team ID: rise_dcd_zju Organization: College of Computer Science and Technology, Zhejiang University ************************************************************* Run ID: rise_dcd_zju1 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.919 0.650 0.761 strong_mention_match 0.882 0.624 0.731 strong_typed_mention_match 0.812 0.575 0.673 strong_typed_all_match 0.822 0.609 0.700 strong_typed_link_match 0.783 0.491 0.604 strong_typed_nil_match 0.805 0.701 0.750 entity_match 0.851 0.602 0.705 mention_ceaf 0.827 0.585 0.685 typed_mention_ceaf 0.790 0.559 0.655 typed_mention_ceaf_plus 0.851 0.508 0.636 b_cubed 0.781 0.477 0.592 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.934 0.655 0.770 strong_mention_match 0.894 0.627 0.737 strong_typed_mention_match 0.819 0.574 0.675 strong_typed_all_match 0.796 0.610 0.691 strong_typed_link_match 0.879 0.504 0.640 strong_typed_nil_match 0.773 0.723 0.747 entity_match 0.848 0.595 0.699 mention_ceaf 0.822 0.576 0.677 typed_mention_ceaf 0.773 0.542 0.637 typed_mention_ceaf_plus 0.847 0.526 0.649 b_cubed 0.762 0.486 0.594 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.906 0.681 0.778 strong_mention_match 0.874 0.657 0.750 strong_typed_mention_match 0.813 0.611 0.698 strong_typed_all_match 0.832 0.679 0.748 strong_typed_link_match 0.730 0.409 0.525 strong_typed_nil_match 0.821 0.748 0.783 entity_match 0.853 0.641 0.732 mention_ceaf 0.830 0.624 0.712 typed_mention_ceaf 0.798 0.600 0.685 typed_mention_ceaf_plus 0.844 0.557 0.671 b_cubed 0.791 0.526 0.632 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.923 0.601 0.728 strong_mention_match 0.884 0.576 0.697 strong_typed_mention_match 0.802 0.523 0.633 strong_typed_all_match 0.830 0.500 0.624 strong_typed_link_match 0.753 0.573 0.651 strong_typed_nil_match 0.816 0.633 0.713 entity_match 0.875 0.570 0.691 mention_ceaf 0.847 0.552 0.669 typed_mention_ceaf 0.796 0.519 0.628 typed_mention_ceaf_plus 0.889 0.459 0.606 b_cubed 0.801 0.427 0.557 b_cubed_plus ************************************************************* Run ID: rise_dcd_zju2 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.909 0.661 0.766 strong_mention_match 0.871 0.634 0.734 strong_typed_mention_match 0.790 0.575 0.665 strong_typed_all_match 0.816 0.606 0.696 strong_typed_link_match 0.721 0.498 0.589 strong_typed_nil_match 0.805 0.694 0.745 entity_match 0.830 0.604 0.699 mention_ceaf 0.805 0.586 0.678 typed_mention_ceaf 0.768 0.559 0.647 typed_mention_ceaf_plus 0.842 0.509 0.634 b_cubed 0.759 0.476 0.585 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.900 0.694 0.784 strong_mention_match 0.856 0.660 0.746 strong_typed_mention_match 0.781 0.603 0.680 strong_typed_all_match 0.753 0.652 0.699 strong_typed_link_match 0.863 0.506 0.638 strong_typed_nil_match 0.729 0.762 0.745 entity_match 0.817 0.630 0.711 mention_ceaf 0.788 0.608 0.686 typed_mention_ceaf 0.739 0.570 0.644 typed_mention_ceaf_plus 0.809 0.568 0.667 b_cubed 0.723 0.521 0.606 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.906 0.681 0.778 strong_mention_match 0.874 0.657 0.750 strong_typed_mention_match 0.813 0.611 0.698 strong_typed_all_match 0.832 0.679 0.748 strong_typed_link_match 0.730 0.409 0.525 strong_typed_nil_match 0.821 0.748 0.783 entity_match 0.853 0.641 0.732 mention_ceaf 0.830 0.624 0.712 typed_mention_ceaf 0.798 0.600 0.685 typed_mention_ceaf_plus 0.844 0.557 0.671 b_cubed 0.791 0.526 0.632 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.923 0.601 0.728 strong_mention_match 0.884 0.576 0.697 strong_typed_mention_match 0.761 0.496 0.600 strong_typed_all_match 0.875 0.452 0.597 strong_typed_link_match 0.622 0.593 0.607 strong_typed_nil_match 0.882 0.579 0.699 entity_match 0.835 0.544 0.659 mention_ceaf 0.808 0.526 0.637 typed_mention_ceaf 0.753 0.491 0.594 typed_mention_ceaf_plus 0.900 0.428 0.580 b_cubed 0.762 0.394 0.519 b_cubed_plus ************************************************************* Run ID: rise_dcd_zju3 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.910 0.694 0.788 strong_mention_match 0.870 0.663 0.752 strong_typed_mention_match 0.767 0.584 0.663 strong_typed_all_match 0.803 0.608 0.692 strong_typed_link_match 0.681 0.528 0.594 strong_typed_nil_match 0.814 0.691 0.747 entity_match 0.801 0.610 0.693 mention_ceaf 0.778 0.593 0.673 typed_mention_ceaf 0.741 0.565 0.641 typed_mention_ceaf_plus 0.806 0.523 0.634 b_cubed 0.718 0.484 0.579 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.905 0.731 0.809 strong_mention_match 0.853 0.689 0.762 strong_typed_mention_match 0.741 0.599 0.663 strong_typed_all_match 0.750 0.616 0.677 strong_typed_link_match 0.723 0.564 0.634 strong_typed_nil_match 0.773 0.723 0.747 entity_match 0.768 0.620 0.686 mention_ceaf 0.741 0.598 0.662 typed_mention_ceaf 0.693 0.559 0.619 typed_mention_ceaf_plus 0.792 0.560 0.656 b_cubed 0.676 0.505 0.578 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.924 0.663 0.772 strong_mention_match 0.891 0.639 0.744 strong_typed_mention_match 0.831 0.596 0.694 strong_typed_all_match 0.850 0.663 0.745 strong_typed_link_match 0.745 0.396 0.517 strong_typed_nil_match 0.821 0.741 0.779 entity_match 0.872 0.626 0.729 mention_ceaf 0.849 0.609 0.709 typed_mention_ceaf 0.817 0.586 0.682 typed_mention_ceaf_plus 0.875 0.539 0.667 b_cubed 0.818 0.510 0.629 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.898 0.703 0.789 strong_mention_match 0.858 0.672 0.754 strong_typed_mention_match 0.708 0.555 0.622 strong_typed_all_match 0.778 0.516 0.620 strong_typed_link_match 0.610 0.641 0.625 strong_typed_nil_match 0.849 0.609 0.710 entity_match 0.763 0.598 0.671 mention_ceaf 0.741 0.580 0.651 typed_mention_ceaf 0.690 0.541 0.607 typed_mention_ceaf_plus 0.753 0.506 0.606 b_cubed 0.646 0.456 0.534 b_cubed_plus ************************************************************* Run ID: rise_dcd_zju4 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.909 0.684 0.781 strong_mention_match 0.871 0.655 0.748 strong_typed_mention_match 0.768 0.578 0.659 strong_typed_all_match 0.808 0.605 0.692 strong_typed_link_match 0.673 0.511 0.581 strong_typed_nil_match 0.805 0.692 0.744 entity_match 0.806 0.606 0.692 mention_ceaf 0.782 0.588 0.671 typed_mention_ceaf 0.744 0.560 0.639 typed_mention_ceaf_plus 0.815 0.515 0.631 b_cubed 0.725 0.478 0.576 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.900 0.694 0.784 strong_mention_match 0.856 0.660 0.746 strong_typed_mention_match 0.781 0.603 0.680 strong_typed_all_match 0.753 0.652 0.699 strong_typed_link_match 0.863 0.506 0.638 strong_typed_nil_match 0.729 0.762 0.745 entity_match 0.817 0.630 0.711 mention_ceaf 0.788 0.608 0.686 typed_mention_ceaf 0.739 0.570 0.644 typed_mention_ceaf_plus 0.809 0.568 0.667 b_cubed 0.723 0.521 0.606 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.924 0.663 0.772 strong_mention_match 0.891 0.639 0.744 strong_typed_mention_match 0.831 0.596 0.694 strong_typed_all_match 0.850 0.663 0.745 strong_typed_link_match 0.745 0.396 0.517 strong_typed_nil_match 0.821 0.741 0.779 entity_match 0.872 0.626 0.729 mention_ceaf 0.849 0.609 0.709 typed_mention_ceaf 0.817 0.586 0.682 typed_mention_ceaf_plus 0.875 0.539 0.667 b_cubed 0.818 0.510 0.629 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.898 0.703 0.789 strong_mention_match 0.858 0.672 0.754 strong_typed_mention_match 0.674 0.528 0.592 strong_typed_all_match 0.796 0.474 0.594 strong_typed_link_match 0.539 0.649 0.589 strong_typed_nil_match 0.882 0.579 0.699 entity_match 0.733 0.574 0.644 mention_ceaf 0.710 0.556 0.624 typed_mention_ceaf 0.656 0.514 0.576 typed_mention_ceaf_plus 0.764 0.475 0.586 b_cubed 0.617 0.423 0.502 b_cubed_plus ************************************************************* Run ID: rise_dcd_zju5 Languages addressed: Chinese, English, Spanish Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.890 0.694 0.780 strong_mention_match 0.847 0.661 0.742 strong_typed_mention_match 0.733 0.572 0.643 strong_typed_all_match 0.774 0.592 0.671 strong_typed_link_match 0.642 0.522 0.576 strong_typed_nil_match 0.802 0.677 0.734 entity_match 0.773 0.603 0.677 mention_ceaf 0.749 0.584 0.657 typed_mention_ceaf 0.711 0.554 0.623 typed_mention_ceaf_plus 0.762 0.513 0.614 b_cubed 0.671 0.471 0.554 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.910 0.663 0.767 strong_mention_match 0.867 0.631 0.730 strong_typed_mention_match 0.798 0.581 0.672 strong_typed_all_match 0.771 0.619 0.686 strong_typed_link_match 0.871 0.506 0.640 strong_typed_nil_match 0.750 0.726 0.738 entity_match 0.829 0.603 0.698 mention_ceaf 0.799 0.582 0.673 typed_mention_ceaf 0.753 0.548 0.634 typed_mention_ceaf_plus 0.823 0.536 0.649 b_cubed 0.742 0.493 0.593 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.872 0.709 0.782 strong_mention_match 0.828 0.673 0.742 strong_typed_mention_match 0.736 0.599 0.660 strong_typed_all_match 0.767 0.656 0.707 strong_typed_link_match 0.623 0.427 0.507 strong_typed_nil_match 0.796 0.736 0.765 entity_match 0.786 0.639 0.705 mention_ceaf 0.763 0.620 0.684 typed_mention_ceaf 0.725 0.589 0.650 typed_mention_ceaf_plus 0.739 0.562 0.639 b_cubed 0.676 0.517 0.586 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.897 0.702 0.788 strong_mention_match 0.858 0.672 0.753 strong_typed_mention_match 0.671 0.525 0.589 strong_typed_all_match 0.792 0.470 0.590 strong_typed_link_match 0.538 0.650 0.589 strong_typed_nil_match 0.876 0.573 0.693 entity_match 0.732 0.573 0.643 mention_ceaf 0.710 0.555 0.623 typed_mention_ceaf 0.652 0.511 0.573 typed_mention_ceaf_plus 0.764 0.473 0.585 b_cubed 0.615 0.420 0.499 b_cubed_plus