=========================================================================== TAC KBP 2017 CROSS-LINGUAL KB CONSTRUCTION: COMPOSITE KB EVALUATION RESULTS =========================================================================== Team ID: A2KD_Adept Organization: Raytheon BBN Technologies ************************************************************* Run ID: A2KD_Adept_KB_XLING_1 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Run number of the English CSKB system that is most closely configured to the English component of this run: 4 Run number of the Spanish CSKB system that is most closely configured to the Spanish component of this run: NA Run number of the Chinese CSKB system that is most closely configured to the Chinese component of this run: 3 Did the run attempt the full Entity Discovery and Linking task: Yes Did the run attempt the full Event Nugget Detection and Coreference task: No Did the run include SF relations: Yes Did the run attempt the Event Argument Extraction and Linking task: Yes Did the run include Sentiment relations: No --------------------- Composite KB Evaluation (All slot types): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0319 SF-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0005 SF-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0293 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0325 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0005 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0299 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0475 SF-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0010 SF-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0434 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0491 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0010 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0451 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_XLING_1 0 0.2761 0.0910 0.1369 SF-ALL-Micro A2KD_Adept_KB_XLING_1 1 0.0082 0.0083 0.0083 SF-ALL-Micro A2KD_Adept_KB_XLING_1 ALL 0.1262 0.0669 0.0874 SF-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0639 0.0643 0.0532 SF-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0031 0.0074 0.0034 SF-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0373 0.0394 0.0314 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_1 0 0.3012 0.1789 0.2245 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_1 1 0.0144 0.0252 0.0183 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_1 ALL 0.1435 0.1340 0.1386 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.1357 0.1366 0.1167 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0069 0.0139 0.0074 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0796 0.0832 0.0691 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0651 0.0659 0.0548 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0031 0.0073 0.0034 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0381 0.0404 0.0324 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SF-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0226 SF-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0007 SF-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0162 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0223 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0007 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0160 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0380 SF-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0016 SF-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0283 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0376 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0016 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0279 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_XLING_1 0 0.2157 0.0616 0.0959 SF-ALL-Micro A2KD_Adept_KB_XLING_1 1 0.0065 0.0146 0.0090 SF-ALL-Micro A2KD_Adept_KB_XLING_1 ALL 0.0637 0.0498 0.0559 SF-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0488 0.0514 0.0397 SF-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0055 0.0137 0.0060 SF-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0277 0.0331 0.0233 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_1 0 0.2468 0.1237 0.1648 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_1 1 0.0077 0.0286 0.0121 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_1 ALL 0.0760 0.0998 0.0863 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.1274 0.1284 0.1072 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0104 0.0240 0.0112 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0708 0.0779 0.0608 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0482 0.0508 0.0393 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0055 0.0137 0.0060 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0276 0.0329 0.0232 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (EVENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0654 SF-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0654 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0673 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0673 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0918 SF-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0918 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0970 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0970 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_XLING_1 0 0.3139 0.1769 0.2263 SF-ALL-Micro A2KD_Adept_KB_XLING_1 ALL 0.3139 0.1769 0.2263 SF-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.1263 0.1245 0.1064 SF-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.1263 0.1245 0.1064 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_1 0 0.3452 0.3447 0.3450 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_1 ALL 0.3452 0.3447 0.3450 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.2397 0.2411 0.2092 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.2397 0.2411 0.2092 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.1305 0.1298 0.1115 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.1305 0.1298 0.1115 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SENTIMENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0000 SF-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0000 SF-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0000 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0000 SF-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0000 SF-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0000 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_XLING_1 0 0.0000 0.0000 0.0000 SF-ALL-Micro A2KD_Adept_KB_XLING_1 1 0.0000 0.0000 0.0000 SF-ALL-Micro A2KD_Adept_KB_XLING_1 ALL 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_1 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_1 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_1 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 0 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_1 ALL 0.0000 0.0000 0.0000 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. ************************************************************* Run ID: A2KD_Adept_KB_XLING_2 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Run number of the English CSKB system that is most closely configured to the English component of this run: 4 Run number of the Spanish CSKB system that is most closely configured to the Spanish component of this run: NA Run number of the Chinese CSKB system that is most closely configured to the Chinese component of this run: 4 Did the run attempt the full Entity Discovery and Linking task: Yes Did the run attempt the full Event Nugget Detection and Coreference task: No Did the run include SF relations: Yes Did the run attempt the Event Argument Extraction and Linking task: Yes Did the run include Sentiment relations: No --------------------- Composite KB Evaluation (All slot types): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0343 SF-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0005 SF-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0306 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0349 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0005 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0312 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0520 SF-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0011 SF-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0446 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0535 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0010 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0462 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_XLING_2 0 0.2431 0.0929 0.1344 SF-ALL-Micro A2KD_Adept_KB_XLING_2 1 0.0063 0.0097 0.0076 SF-ALL-Micro A2KD_Adept_KB_XLING_2 ALL 0.0952 0.0686 0.0798 SF-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0636 0.0688 0.0543 SF-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0035 0.0089 0.0040 SF-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0373 0.0426 0.0323 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_2 0 0.2723 0.1817 0.2180 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_2 1 0.0137 0.0275 0.0183 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_2 ALL 0.1290 0.1367 0.1327 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.1336 0.1426 0.1168 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0074 0.0165 0.0083 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0787 0.0877 0.0696 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0648 0.0704 0.0559 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0034 0.0088 0.0039 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0381 0.0436 0.0333 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SF-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0283 SF-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0007 SF-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0194 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0280 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0007 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0192 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0488 SF-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0019 SF-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0311 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0482 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0019 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0307 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_XLING_2 0 0.1633 0.0659 0.0939 SF-ALL-Micro A2KD_Adept_KB_XLING_2 1 0.0053 0.0184 0.0083 SF-ALL-Micro A2KD_Adept_KB_XLING_2 ALL 0.0460 0.0539 0.0496 SF-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0482 0.0622 0.0424 SF-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0062 0.0168 0.0071 SF-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0277 0.0401 0.0252 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_2 0 0.1773 0.1301 0.1501 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_2 1 0.0086 0.0349 0.0138 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_2 ALL 0.0677 0.1061 0.0826 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.1223 0.1430 0.1073 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0115 0.0291 0.0130 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0687 0.0879 0.0617 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0477 0.0615 0.0419 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0062 0.0168 0.0071 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0276 0.0399 0.0251 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (EVENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0654 SF-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0654 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0672 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0672 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0918 SF-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0918 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0970 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0970 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_XLING_2 0 0.3139 0.1769 0.2263 SF-ALL-Micro A2KD_Adept_KB_XLING_2 ALL 0.3139 0.1769 0.2263 SF-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.1263 0.1245 0.1064 SF-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.1263 0.1245 0.1064 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_2 0 0.3429 0.3447 0.3438 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_2 ALL 0.3429 0.3447 0.3438 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.2397 0.2411 0.2092 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.2397 0.2411 0.2092 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.1305 0.1298 0.1115 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.1305 0.1298 0.1115 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SENTIMENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0000 SF-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0000 SF-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0000 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0000 SF-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0000 SF-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0000 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro A2KD_Adept_KB_XLING_2 0 0.0000 0.0000 0.0000 SF-ALL-Micro A2KD_Adept_KB_XLING_2 1 0.0000 0.0000 0.0000 SF-ALL-Micro A2KD_Adept_KB_XLING_2 ALL 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0000 0.0000 0.0000 SF-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_2 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro A2KD_Adept_KB_XLING_2 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 0 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro A2KD_Adept_KB_XLING_2 ALL 0.0000 0.0000 0.0000 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1.