======================================================================= TAC KBP 2015 DIAGNOSTIC TRILINGUAL ENTITY LINKING EVALUATION RESULTS ======================================================================= Team ID: BUPTTeam Organization: Beijing University of Posts and Telecommunications ************************************************************* Run ID: BUPTTeam1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.987 0.924 0.954 strong_mention_match 0.897 0.840 0.868 strong_typed_mention_match 0.735 0.689 0.711 strong_typed_all_match 0.721 0.686 0.703 strong_typed_link_match 0.778 0.696 0.734 strong_typed_nil_match 0.629 0.710 0.667 entity_match 0.735 0.689 0.711 mention_ceaf 0.700 0.656 0.677 typed_mention_ceaf 0.651 0.610 0.630 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.971 0.891 0.930 strong_mention_match 0.886 0.813 0.848 strong_typed_mention_match 0.692 0.635 0.662 strong_typed_all_match 0.669 0.657 0.663 strong_typed_link_match 0.769 0.580 0.661 strong_typed_nil_match 0.573 0.728 0.642 entity_match 0.801 0.735 0.766 mention_ceaf 0.757 0.694 0.724 typed_mention_ceaf 0.677 0.621 0.648 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 1.000 0.947 0.973 strong_mention_match 0.944 0.894 0.918 strong_typed_mention_match 0.854 0.809 0.831 strong_typed_all_match 0.843 0.807 0.825 strong_typed_link_match 0.896 0.814 0.853 strong_typed_nil_match 0.797 0.767 0.782 entity_match 0.929 0.880 0.904 mention_ceaf 0.893 0.846 0.869 typed_mention_ceaf 0.838 0.794 0.815 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 1.000 0.966 0.983 strong_mention_match 0.838 0.810 0.824 strong_typed_mention_match 0.622 0.601 0.612 strong_typed_all_match 0.598 0.513 0.552 strong_typed_link_match 0.666 0.827 0.738 strong_typed_nil_match 0.559 0.601 0.579 entity_match 0.726 0.701 0.713 mention_ceaf 0.661 0.638 0.649 typed_mention_ceaf 0.570 0.550 0.560 typed_mention_ceaf_plus ************************************************************* Run ID: BUPTTeam2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.987 0.926 0.955 strong_mention_match 0.897 0.842 0.869 strong_typed_mention_match 0.735 0.690 0.712 strong_typed_all_match 0.721 0.687 0.703 strong_typed_link_match 0.778 0.699 0.737 strong_typed_nil_match 0.629 0.711 0.668 entity_match 0.734 0.689 0.711 mention_ceaf 0.700 0.656 0.677 typed_mention_ceaf 0.651 0.610 0.630 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.971 0.894 0.931 strong_mention_match 0.886 0.816 0.850 strong_typed_mention_match 0.692 0.637 0.664 strong_typed_all_match 0.668 0.657 0.663 strong_typed_link_match 0.769 0.588 0.666 strong_typed_nil_match 0.573 0.729 0.642 entity_match 0.798 0.735 0.765 mention_ceaf 0.755 0.695 0.724 typed_mention_ceaf 0.675 0.622 0.647 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 1.000 0.947 0.973 strong_mention_match 0.944 0.894 0.919 strong_typed_mention_match 0.854 0.809 0.831 strong_typed_all_match 0.843 0.807 0.825 strong_typed_link_match 0.896 0.814 0.853 strong_typed_nil_match 0.797 0.768 0.782 entity_match 0.929 0.880 0.904 mention_ceaf 0.893 0.846 0.869 typed_mention_ceaf 0.838 0.794 0.815 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 1.000 0.966 0.983 strong_mention_match 0.838 0.810 0.824 strong_typed_mention_match 0.622 0.601 0.612 strong_typed_all_match 0.598 0.513 0.552 strong_typed_link_match 0.666 0.827 0.738 strong_typed_nil_match 0.559 0.601 0.579 entity_match 0.726 0.701 0.713 mention_ceaf 0.661 0.638 0.649 typed_mention_ceaf 0.570 0.550 0.560 typed_mention_ceaf_plus ************************************************************* Run ID: BUPTTeam3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: No Did the run use the relation/slot information encoded in the Reference KB: No Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.987 0.923 0.954 strong_mention_match 0.897 0.840 0.868 strong_typed_mention_match 0.736 0.688 0.711 strong_typed_all_match 0.722 0.686 0.703 strong_typed_link_match 0.778 0.696 0.734 strong_typed_nil_match 0.629 0.709 0.667 entity_match 0.735 0.688 0.711 mention_ceaf 0.700 0.655 0.677 typed_mention_ceaf 0.652 0.610 0.630 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.971 0.891 0.929 strong_mention_match 0.886 0.813 0.848 strong_typed_mention_match 0.692 0.635 0.662 strong_typed_all_match 0.669 0.657 0.663 strong_typed_link_match 0.769 0.580 0.661 strong_typed_nil_match 0.573 0.728 0.641 entity_match 0.801 0.734 0.766 mention_ceaf 0.757 0.694 0.724 typed_mention_ceaf 0.677 0.621 0.648 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 1.000 0.947 0.973 strong_mention_match 0.944 0.893 0.918 strong_typed_mention_match 0.854 0.809 0.831 strong_typed_all_match 0.843 0.807 0.825 strong_typed_link_match 0.896 0.814 0.853 strong_typed_nil_match 0.797 0.767 0.781 entity_match 0.929 0.880 0.904 mention_ceaf 0.893 0.845 0.868 typed_mention_ceaf 0.838 0.794 0.815 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 1.000 0.964 0.982 strong_mention_match 0.838 0.808 0.823 strong_typed_mention_match 0.622 0.599 0.610 strong_typed_all_match 0.596 0.510 0.550 strong_typed_link_match 0.666 0.827 0.738 strong_typed_nil_match 0.558 0.599 0.578 entity_match 0.725 0.699 0.712 mention_ceaf 0.660 0.636 0.648 typed_mention_ceaf 0.569 0.548 0.558 typed_mention_ceaf_plus