==================================================== TAC KBP 2017 ENGLISH SLOT FILLING EVALUATION RESULTS ==================================================== Team ID: Y_dcd_zju Organization: Zhejiang University ************************************************************* Run ID: Y_dcd_zju_SF_ENG_1 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation (Queries involve ONLY SF slots): Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro Y_dcd_zju_SF_ENG_1 0 0.2585 0.0924 0.1361 SF-ALL-Micro Y_dcd_zju_SF_ENG_1 1 0.0074 0.0137 0.0096 SF-ALL-Micro Y_dcd_zju_SF_ENG_1 ALL 0.1025 0.0735 0.0856 SF-ALL-Macro Y_dcd_zju_SF_ENG_1 0 0.1198 0.1136 0.1026 SF-ALL-Macro Y_dcd_zju_SF_ENG_1 1 0.0091 0.0095 0.0093 SF-ALL-Macro Y_dcd_zju_SF_ENG_1 ALL 0.0759 0.0724 0.0656 LDC-MAX-ALL-Micro Y_dcd_zju_SF_ENG_1 0 0.2489 0.1069 0.1495 LDC-MAX-ALL-Micro Y_dcd_zju_SF_ENG_1 1 0.0060 0.0127 0.0081 LDC-MAX-ALL-Micro Y_dcd_zju_SF_ENG_1 ALL 0.1036 0.0850 0.0934 LDC-MAX-ALL-Macro Y_dcd_zju_SF_ENG_1 0 0.1343 0.1371 0.1181 LDC-MAX-ALL-Macro Y_dcd_zju_SF_ENG_1 1 0.0106 0.0110 0.0108 LDC-MAX-ALL-Macro Y_dcd_zju_SF_ENG_1 ALL 0.0863 0.0881 0.0765 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_1 0 0.1297 0.1300 0.1127 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_1 1 0.0106 0.0110 0.0108 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_1 ALL 0.0835 0.0838 0.0731 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro Y_dcd_zju_SF_ENG_1 0 0.0947 SF-ALL-Macro Y_dcd_zju_SF_ENG_1 1 0.0016 SF-ALL-Macro Y_dcd_zju_SF_ENG_1 ALL 0.0800 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_1 0 0.1032 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_1 1 0.0013 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_1 ALL 0.0868 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. ************************************************************* Run ID: Y_dcd_zju_SF_ENG_2 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation (Queries involve ONLY SF slots): Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro Y_dcd_zju_SF_ENG_2 0 0.0510 0.1079 0.0693 SF-ALL-Micro Y_dcd_zju_SF_ENG_2 1 0.0041 0.0219 0.0069 SF-ALL-Micro Y_dcd_zju_SF_ENG_2 ALL 0.0302 0.0873 0.0449 SF-ALL-Macro Y_dcd_zju_SF_ENG_2 0 0.1209 0.1531 0.1088 SF-ALL-Macro Y_dcd_zju_SF_ENG_2 1 0.0159 0.0172 0.0138 SF-ALL-Macro Y_dcd_zju_SF_ENG_2 ALL 0.0793 0.0992 0.0711 LDC-MAX-ALL-Micro Y_dcd_zju_SF_ENG_2 0 0.0570 0.1317 0.0796 LDC-MAX-ALL-Micro Y_dcd_zju_SF_ENG_2 1 0.0036 0.0253 0.0063 LDC-MAX-ALL-Micro Y_dcd_zju_SF_ENG_2 ALL 0.0314 0.1070 0.0485 LDC-MAX-ALL-Macro Y_dcd_zju_SF_ENG_2 0 0.1468 0.2089 0.1397 LDC-MAX-ALL-Macro Y_dcd_zju_SF_ENG_2 1 0.0199 0.0252 0.0175 LDC-MAX-ALL-Macro Y_dcd_zju_SF_ENG_2 ALL 0.0975 0.1376 0.0923 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_2 0 0.1181 0.1743 0.1099 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_2 1 0.0199 0.0252 0.0175 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_2 ALL 0.0799 0.1164 0.0740 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro Y_dcd_zju_SF_ENG_2 0 0.1167 SF-ALL-Macro Y_dcd_zju_SF_ENG_2 1 0.0064 SF-ALL-Macro Y_dcd_zju_SF_ENG_2 ALL 0.0976 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_2 0 0.1265 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_2 1 0.0078 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_2 ALL 0.1068 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. ************************************************************* Run ID: Y_dcd_zju_SF_ENG_3 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation (Queries involve ONLY SF slots): Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro Y_dcd_zju_SF_ENG_3 0 0.0899 0.0656 0.0759 SF-ALL-Micro Y_dcd_zju_SF_ENG_3 1 0.0000 0.0000 0.0000 SF-ALL-Micro Y_dcd_zju_SF_ENG_3 ALL 0.0665 0.0499 0.0570 SF-ALL-Macro Y_dcd_zju_SF_ENG_3 0 0.0864 0.0831 0.0698 SF-ALL-Macro Y_dcd_zju_SF_ENG_3 1 0.0000 0.0000 0.0000 SF-ALL-Macro Y_dcd_zju_SF_ENG_3 ALL 0.0521 0.0502 0.0421 LDC-MAX-ALL-Micro Y_dcd_zju_SF_ENG_3 0 0.1092 0.0840 0.0949 LDC-MAX-ALL-Micro Y_dcd_zju_SF_ENG_3 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro Y_dcd_zju_SF_ENG_3 ALL 0.0698 0.0645 0.0671 LDC-MAX-ALL-Macro Y_dcd_zju_SF_ENG_3 0 0.0973 0.1068 0.0815 LDC-MAX-ALL-Macro Y_dcd_zju_SF_ENG_3 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro Y_dcd_zju_SF_ENG_3 ALL 0.0595 0.0653 0.0499 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_3 0 0.0885 0.0957 0.0720 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_3 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_3 ALL 0.0541 0.0585 0.0440 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro Y_dcd_zju_SF_ENG_3 0 0.0679 SF-ALL-Macro Y_dcd_zju_SF_ENG_3 1 0.0000 SF-ALL-Macro Y_dcd_zju_SF_ENG_3 ALL 0.0636 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_3 0 0.0713 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_3 1 0.0000 LDC-MEAN-ALL-Macro Y_dcd_zju_SF_ENG_3 ALL 0.0659 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values.