===================================================================== TAC KBP 2017 CHINESE KB CONSTRUCTION: COMPOSITE KB EVALUATION RESULTS ===================================================================== Team ID: A2KD_Adept Organization: Raytheon BBN Technologies ************************************************************* Run ID: A2KD_Adept_KB_CMN_1 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Did the run attempt the full Entity Discovery and Linking task: Yes Did the run attempt the full Event Nugget Detection and Coreference task: No Did the run include SF relations: Yes Did the run attempt the Event Argument Extraction and Linking task: Yes Did the run include Sentiment relations: No --------------------- Composite KB Evaluation (All slot types): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.0729 SF-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0096 SF-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0646 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.0742 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0070 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0666 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.1028 SF-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0034 SF-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0892 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.1046 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0049 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0934 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_1 0 0.2548 0.1457 0.1854 SF-ALL-Micro A2KD_Adept_KB_CMN_1 1 0.0095 0.0231 0.0135 SF-ALL-Micro A2KD_Adept_KB_CMN_1 ALL 0.0934 0.1075 0.1000 SF-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.1167 0.1285 0.1064 SF-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0099 0.0218 0.0116 SF-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0769 0.0888 0.0711 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_1 0 0.2692 0.1734 0.2110 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_1 1 0.0142 0.0354 0.0202 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_1 ALL 0.1051 0.1296 0.1161 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.1371 0.1461 0.1260 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0173 0.0382 0.0203 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0953 0.1084 0.0890 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.1243 0.1311 0.1126 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0164 0.0362 0.0190 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0866 0.0979 0.0799 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SF-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.0774 SF-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0167 SF-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0601 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.0824 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0100 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0669 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.1064 SF-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0039 SF-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0782 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.1174 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0048 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0947 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_1 0 0.2057 0.1868 0.1958 SF-ALL-Micro A2KD_Adept_KB_CMN_1 1 0.0071 0.0663 0.0128 SF-ALL-Micro A2KD_Adept_KB_CMN_1 ALL 0.0568 0.1596 0.0838 SF-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.0990 0.1415 0.1002 SF-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0194 0.0482 0.0237 SF-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0719 0.1097 0.0741 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_1 0 0.2130 0.2102 0.2116 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_1 1 0.0080 0.1053 0.0148 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_1 ALL 0.0548 0.1890 0.0850 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.1221 0.1611 0.1216 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0280 0.0695 0.0341 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0884 0.1283 0.0903 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.1174 0.1542 0.1164 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0263 0.0656 0.0317 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0847 0.1224 0.0861 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (EVENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.1062 SF-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.1062 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.1020 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.1020 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.1538 SF-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.1538 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.1417 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.1417 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_1 0 0.3306 0.2165 0.2616 SF-ALL-Micro A2KD_Adept_KB_CMN_1 ALL 0.3306 0.2165 0.2616 SF-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.2059 0.1802 0.1736 SF-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.2059 0.1802 0.1736 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_1 0 0.3429 0.2414 0.2833 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_1 ALL 0.3429 0.2414 0.2833 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.2332 0.2026 0.2002 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.2332 0.2026 0.2002 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.2014 0.1673 0.1674 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.2014 0.1673 0.1674 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SENTIMENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0000 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0000 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_1 0 0.0000 0.0000 0.0000 SF-ALL-Micro A2KD_Adept_KB_CMN_1 1 0.0000 0.0000 0.0000 SF-ALL-Micro A2KD_Adept_KB_CMN_1 ALL 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_1 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_1 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_1 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 0 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_1 ALL 0.0000 0.0000 0.0000 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. ************************************************************* Run ID: A2KD_Adept_KB_CMN_2 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Did the run attempt the full Entity Discovery and Linking task: Yes Did the run attempt the full Event Nugget Detection and Coreference task: No Did the run include SF relations: Yes Did the run attempt the Event Argument Extraction and Linking task: Yes Did the run include Sentiment relations: No --------------------- Composite KB Evaluation (All slot types): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.0834 SF-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0169 SF-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0718 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.0821 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0105 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0725 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.1104 SF-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0050 SF-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0936 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.1145 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0075 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0993 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_2 0 0.1912 0.1487 0.1673 SF-ALL-Micro A2KD_Adept_KB_CMN_2 1 0.0055 0.0231 0.0089 SF-ALL-Micro A2KD_Adept_KB_CMN_2 ALL 0.0596 0.1097 0.0772 SF-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.1131 0.1364 0.1033 SF-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0096 0.0218 0.0112 SF-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0746 0.0937 0.0690 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_2 0 0.2238 0.1783 0.1985 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_2 1 0.0095 0.0354 0.0150 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_2 ALL 0.0770 0.1329 0.0975 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.1374 0.1587 0.1286 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0169 0.0382 0.0197 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0953 0.1166 0.0905 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.1249 0.1436 0.1156 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0160 0.0362 0.0184 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0868 0.1061 0.0816 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SF-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.1000 SF-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0313 SF-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0759 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.0990 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0170 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0794 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.1228 SF-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0070 SF-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0879 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.1381 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0100 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.1073 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_2 0 0.1344 0.1948 0.1591 SF-ALL-Micro A2KD_Adept_KB_CMN_2 1 0.0042 0.0663 0.0078 SF-ALL-Micro A2KD_Adept_KB_CMN_2 ALL 0.0351 0.1658 0.0580 SF-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.0915 0.1579 0.0936 SF-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0189 0.0482 0.0228 SF-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0668 0.1205 0.0695 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_2 0 0.1595 0.2212 0.1854 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_2 1 0.0056 0.1053 0.0106 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_2 ALL 0.0402 0.1979 0.0669 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.1227 0.1866 0.1269 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0272 0.0695 0.0329 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0885 0.1446 0.0932 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.1186 0.1797 0.1226 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0254 0.0656 0.0305 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0853 0.1388 0.0896 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (EVENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.1052 SF-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.1052 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.1012 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.1012 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.1528 SF-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.1528 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.1409 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.1409 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_2 0 0.3306 0.2165 0.2616 SF-ALL-Micro A2KD_Adept_KB_CMN_2 ALL 0.3306 0.2165 0.2616 SF-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.2059 0.1802 0.1736 SF-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.2059 0.1802 0.1736 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_2 0 0.3401 0.2414 0.2824 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_2 ALL 0.3401 0.2414 0.2824 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.2332 0.2026 0.2002 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.2332 0.2026 0.2002 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.2014 0.1673 0.1674 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.2014 0.1673 0.1674 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SENTIMENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0000 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0000 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_2 0 0.0000 0.0000 0.0000 SF-ALL-Micro A2KD_Adept_KB_CMN_2 1 0.0000 0.0000 0.0000 SF-ALL-Micro A2KD_Adept_KB_CMN_2 ALL 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_2 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_2 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 0 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_2 ALL 0.0000 0.0000 0.0000 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. ************************************************************* Run ID: A2KD_Adept_KB_CMN_3 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Did the run attempt the full Entity Discovery and Linking task: Yes Did the run attempt the full Event Nugget Detection and Coreference task: No Did the run include SF relations: Yes Did the run attempt the Event Argument Extraction and Linking task: Yes Did the run include Sentiment relations: No --------------------- Composite KB Evaluation (All slot types): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.0831 SF-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0024 SF-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.0730 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.0917 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0027 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.0824 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.1159 SF-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0043 SF-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.1014 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.1262 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0055 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.1136 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_3 0 0.2697 0.1616 0.2022 SF-ALL-Micro A2KD_Adept_KB_CMN_3 1 0.0099 0.0218 0.0136 SF-ALL-Micro A2KD_Adept_KB_CMN_3 ALL 0.1077 0.1181 0.1127 SF-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.1318 0.1436 0.1176 SF-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0095 0.0223 0.0109 SF-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.0863 0.0984 0.0779 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_3 0 0.2748 0.1986 0.2306 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_3 1 0.0088 0.0271 0.0133 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_3 ALL 0.0979 0.1442 0.1166 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.1612 0.1734 0.1470 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0159 0.0292 0.0178 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.1104 0.1230 0.1018 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.1448 0.1565 0.1317 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0159 0.0286 0.0177 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.0997 0.1118 0.0918 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SF-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.0962 SF-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0025 SF-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.0751 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.1137 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0016 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.0949 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.1330 SF-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0056 SF-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.1028 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.1571 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0062 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.1316 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_3 0 0.2198 0.2110 0.2153 SF-ALL-Micro A2KD_Adept_KB_CMN_3 1 0.0073 0.0608 0.0130 SF-ALL-Micro A2KD_Adept_KB_CMN_3 ALL 0.0673 0.1771 0.0975 SF-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.1184 0.1674 0.1202 SF-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0185 0.0495 0.0220 SF-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.0844 0.1273 0.0867 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_3 0 0.2402 0.2434 0.2418 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_3 1 0.0080 0.0702 0.0144 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_3 ALL 0.0813 0.2085 0.1169 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.1522 0.2014 0.1527 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0253 0.0519 0.0292 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.1068 0.1479 0.1085 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.1463 0.1944 0.1471 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0252 0.0506 0.0290 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.1029 0.1429 0.1048 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (EVENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.1099 SF-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.1099 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.1083 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.1083 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.1548 SF-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.1548 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.1481 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.1481 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_3 0 0.3483 0.2361 0.2814 SF-ALL-Micro A2KD_Adept_KB_CMN_3 ALL 0.3483 0.2361 0.2814 SF-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.2231 0.1878 0.1781 SF-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.2231 0.1878 0.1781 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_3 0 0.3626 0.2730 0.3115 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_3 ALL 0.3626 0.2730 0.3115 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.2614 0.2250 0.2176 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.2614 0.2250 0.2176 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.2205 0.1844 0.1796 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.2205 0.1844 0.1796 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SENTIMENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.0000 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.0000 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_3 0 0.0000 0.0000 0.0000 SF-ALL-Micro A2KD_Adept_KB_CMN_3 1 0.0000 0.0000 0.0000 SF-ALL-Micro A2KD_Adept_KB_CMN_3 ALL 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_3 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_3 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_3 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 0 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_3 ALL 0.0000 0.0000 0.0000 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. ************************************************************* Run ID: A2KD_Adept_KB_CMN_4 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Did the run attempt the full Entity Discovery and Linking task: Yes Did the run attempt the full Event Nugget Detection and Coreference task: No Did the run include SF relations: Yes Did the run attempt the Event Argument Extraction and Linking task: Yes Did the run include Sentiment relations: No --------------------- Composite KB Evaluation (All slot types): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.0918 SF-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0021 SF-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.0806 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1007 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0027 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.0899 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1253 SF-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0040 SF-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.1069 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1386 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0056 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.1212 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_4 0 0.2036 0.1653 0.1825 SF-ALL-Micro A2KD_Adept_KB_CMN_4 1 0.0074 0.0313 0.0120 SF-ALL-Micro A2KD_Adept_KB_CMN_4 ALL 0.0660 0.1236 0.0860 SF-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1281 0.1522 0.1146 SF-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0126 0.0374 0.0157 SF-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.0851 0.1095 0.0778 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_4 0 0.2252 0.2045 0.2143 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_4 1 0.0071 0.0333 0.0117 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_4 ALL 0.0710 0.1501 0.0965 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1616 0.1871 0.1501 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0176 0.0425 0.0204 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.1112 0.1366 0.1047 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1455 0.1702 0.1352 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0175 0.0418 0.0203 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.1008 0.1253 0.0950 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SF-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1141 SF-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0019 SF-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.0910 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1321 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0016 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.1102 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1525 SF-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0052 SF-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.1142 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1826 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0063 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.1471 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_4 0 0.1454 0.2206 0.1753 SF-ALL-Micro A2KD_Adept_KB_CMN_4 1 0.0061 0.0994 0.0115 SF-ALL-Micro A2KD_Adept_KB_CMN_4 ALL 0.0399 0.1933 0.0661 SF-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1108 0.1854 0.1140 SF-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0259 0.0856 0.0336 SF-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.0819 0.1514 0.0866 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_4 0 0.1719 0.2566 0.2059 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_4 1 0.0052 0.0965 0.0099 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_4 ALL 0.0456 0.2244 0.0758 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1528 0.2294 0.1589 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0286 0.0779 0.0344 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.1083 0.1751 0.1143 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1477 0.2223 0.1542 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0285 0.0766 0.0343 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.1050 0.1702 0.1112 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (EVENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1099 SF-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.1099 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1082 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.1082 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1548 SF-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.1548 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.1480 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.1480 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_4 0 0.3483 0.2361 0.2814 SF-ALL-Micro A2KD_Adept_KB_CMN_4 ALL 0.3483 0.2361 0.2814 SF-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.2231 0.1878 0.1781 SF-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.2231 0.1878 0.1781 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_4 0 0.3612 0.2730 0.3110 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_4 ALL 0.3612 0.2730 0.3110 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.2614 0.2250 0.2176 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.2614 0.2250 0.2176 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.2205 0.1844 0.1796 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.2205 0.1844 0.1796 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SENTIMENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.0000 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.0000 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_CMN_4 0 0.0000 0.0000 0.0000 SF-ALL-Micro A2KD_Adept_KB_CMN_4 1 0.0000 0.0000 0.0000 SF-ALL-Micro A2KD_Adept_KB_CMN_4 ALL 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_4 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_4 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_CMN_4 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 0 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_CMN_4 ALL 0.0000 0.0000 0.0000 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1.