===================================================================== TAC KBP 2017 SPANISH KB CONSTRUCTION: COMPOSITE KB EVALUATION RESULTS ===================================================================== Team ID: STANFORD Organization: Stanford University ************************************************************* Run ID: STANFORD_KB_SPA_1 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Did the run attempt the full Entity Discovery and Linking task: No Did the run attempt the full Event Nugget Detection and Coreference task: No Did the run include SF relations: Yes Did the run attempt the Event Argument Extraction and Linking task: No Did the run include Sentiment relations: No --------------------- Composite KB Evaluation (All slot types): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_1 0 0.0960 SF-ALL-Macro STANFORD_KB_SPA_1 1 0.0185 SF-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0661 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 0 0.1038 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 1 0.0161 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0700 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_1 0 0.1210 SF-ALL-Macro STANFORD_KB_SPA_1 1 0.0272 SF-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0860 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 0 0.1340 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 1 0.0236 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0931 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro STANFORD_KB_SPA_1 0 0.2286 0.1324 0.1676 SF-ALL-Micro STANFORD_KB_SPA_1 1 0.4000 0.0563 0.0988 SF-ALL-Micro STANFORD_KB_SPA_1 ALL 0.2479 0.1063 0.1488 SF-ALL-Macro STANFORD_KB_SPA_1 0 0.1223 0.1282 0.1176 SF-ALL-Macro STANFORD_KB_SPA_1 1 0.0653 0.0653 0.0637 SF-ALL-Macro STANFORD_KB_SPA_1 ALL 0.1010 0.1048 0.0974 LDC-MAX-ALL-Micro STANFORD_KB_SPA_1 0 0.1895 0.1610 0.1741 LDC-MAX-ALL-Micro STANFORD_KB_SPA_1 1 0.2963 0.0485 0.0833 LDC-MAX-ALL-Micro STANFORD_KB_SPA_1 ALL 0.2000 0.1204 0.1503 LDC-MAX-ALL-Macro STANFORD_KB_SPA_1 0 0.1460 0.1541 0.1429 LDC-MAX-ALL-Macro STANFORD_KB_SPA_1 1 0.0586 0.0586 0.0571 LDC-MAX-ALL-Macro STANFORD_KB_SPA_1 ALL 0.1141 0.1192 0.1115 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 0 0.1356 0.1403 0.1307 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 1 0.0586 0.0586 0.0571 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 ALL 0.1075 0.1105 0.1038 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SF-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_1 0 0.1743 SF-ALL-Macro STANFORD_KB_SPA_1 1 0.0389 SF-ALL-Macro STANFORD_KB_SPA_1 ALL 0.1200 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 0 0.1822 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 1 0.0328 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 ALL 0.1228 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_1 0 0.2196 SF-ALL-Macro STANFORD_KB_SPA_1 1 0.0572 SF-ALL-Macro STANFORD_KB_SPA_1 ALL 0.1561 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 0 0.2352 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 1 0.0483 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 ALL 0.1634 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro STANFORD_KB_SPA_1 0 0.2286 0.2759 0.2500 SF-ALL-Micro STANFORD_KB_SPA_1 1 0.4000 0.1032 0.1641 SF-ALL-Micro STANFORD_KB_SPA_1 ALL 0.2479 0.2115 0.2283 SF-ALL-Macro STANFORD_KB_SPA_1 0 0.2220 0.2328 0.2134 SF-ALL-Macro STANFORD_KB_SPA_1 1 0.1215 0.1215 0.1184 SF-ALL-Macro STANFORD_KB_SPA_1 ALL 0.1851 0.1919 0.1785 LDC-MAX-ALL-Micro STANFORD_KB_SPA_1 0 0.1888 0.3133 0.2356 LDC-MAX-ALL-Micro STANFORD_KB_SPA_1 1 0.2963 0.0860 0.1333 LDC-MAX-ALL-Micro STANFORD_KB_SPA_1 ALL 0.1993 0.2263 0.2119 LDC-MAX-ALL-Macro STANFORD_KB_SPA_1 0 0.2561 0.2705 0.2506 LDC-MAX-ALL-Macro STANFORD_KB_SPA_1 1 0.1083 0.1083 0.1056 LDC-MAX-ALL-Macro STANFORD_KB_SPA_1 ALL 0.2040 0.2132 0.1994 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 0 0.2380 0.2462 0.2293 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 1 0.1083 0.1083 0.1056 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 ALL 0.1922 0.1975 0.1856 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (EVENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_1 0 0.0000 SF-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 0 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0000 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_1 0 0.0000 SF-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 0 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0000 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro STANFORD_KB_SPA_1 0 0.0000 0.0000 0.0000 SF-ALL-Micro STANFORD_KB_SPA_1 ALL 0.0000 0.0000 0.0000 SF-ALL-Macro STANFORD_KB_SPA_1 0 0.0000 0.0000 0.0000 SF-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro STANFORD_KB_SPA_1 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro STANFORD_KB_SPA_1 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro STANFORD_KB_SPA_1 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 0 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0000 0.0000 0.0000 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SENTIMENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_1 0 0.0000 SF-ALL-Macro STANFORD_KB_SPA_1 1 0.0000 SF-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 0 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 1 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0000 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_1 0 0.0000 SF-ALL-Macro STANFORD_KB_SPA_1 1 0.0000 SF-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 0 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 1 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0000 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro STANFORD_KB_SPA_1 0 0.0000 0.0000 0.0000 SF-ALL-Micro STANFORD_KB_SPA_1 1 0.0000 0.0000 0.0000 SF-ALL-Micro STANFORD_KB_SPA_1 ALL 0.0000 0.0000 0.0000 SF-ALL-Macro STANFORD_KB_SPA_1 0 0.0000 0.0000 0.0000 SF-ALL-Macro STANFORD_KB_SPA_1 1 0.0000 0.0000 0.0000 SF-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro STANFORD_KB_SPA_1 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro STANFORD_KB_SPA_1 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro STANFORD_KB_SPA_1 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro STANFORD_KB_SPA_1 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro STANFORD_KB_SPA_1 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 0 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_1 ALL 0.0000 0.0000 0.0000 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. ************************************************************* Run ID: STANFORD_KB_SPA_2 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Did the run attempt the full Entity Discovery and Linking task: No Did the run attempt the full Event Nugget Detection and Coreference task: No Did the run include SF relations: Yes Did the run attempt the Event Argument Extraction and Linking task: No Did the run include Sentiment relations: No --------------------- Composite KB Evaluation (All slot types): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_2 0 0.0960 SF-ALL-Macro STANFORD_KB_SPA_2 1 0.0185 SF-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0661 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 0 0.1038 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 1 0.0161 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0700 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_2 0 0.1210 SF-ALL-Macro STANFORD_KB_SPA_2 1 0.0272 SF-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0860 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 0 0.1340 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 1 0.0236 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0931 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro STANFORD_KB_SPA_2 0 0.2286 0.1324 0.1676 SF-ALL-Micro STANFORD_KB_SPA_2 1 0.4000 0.0563 0.0988 SF-ALL-Micro STANFORD_KB_SPA_2 ALL 0.2479 0.1063 0.1488 SF-ALL-Macro STANFORD_KB_SPA_2 0 0.1223 0.1282 0.1176 SF-ALL-Macro STANFORD_KB_SPA_2 1 0.0653 0.0653 0.0637 SF-ALL-Macro STANFORD_KB_SPA_2 ALL 0.1010 0.1048 0.0974 LDC-MAX-ALL-Micro STANFORD_KB_SPA_2 0 0.1895 0.1610 0.1741 LDC-MAX-ALL-Micro STANFORD_KB_SPA_2 1 0.2963 0.0485 0.0833 LDC-MAX-ALL-Micro STANFORD_KB_SPA_2 ALL 0.2000 0.1204 0.1503 LDC-MAX-ALL-Macro STANFORD_KB_SPA_2 0 0.1460 0.1541 0.1429 LDC-MAX-ALL-Macro STANFORD_KB_SPA_2 1 0.0586 0.0586 0.0571 LDC-MAX-ALL-Macro STANFORD_KB_SPA_2 ALL 0.1141 0.1192 0.1115 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 0 0.1356 0.1403 0.1307 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 1 0.0586 0.0586 0.0571 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 ALL 0.1075 0.1105 0.1038 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SF-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_2 0 0.1743 SF-ALL-Macro STANFORD_KB_SPA_2 1 0.0389 SF-ALL-Macro STANFORD_KB_SPA_2 ALL 0.1200 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 0 0.1822 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 1 0.0328 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 ALL 0.1228 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_2 0 0.2196 SF-ALL-Macro STANFORD_KB_SPA_2 1 0.0572 SF-ALL-Macro STANFORD_KB_SPA_2 ALL 0.1561 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 0 0.2352 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 1 0.0483 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 ALL 0.1634 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro STANFORD_KB_SPA_2 0 0.2286 0.2759 0.2500 SF-ALL-Micro STANFORD_KB_SPA_2 1 0.4000 0.1032 0.1641 SF-ALL-Micro STANFORD_KB_SPA_2 ALL 0.2479 0.2115 0.2283 SF-ALL-Macro STANFORD_KB_SPA_2 0 0.2220 0.2328 0.2134 SF-ALL-Macro STANFORD_KB_SPA_2 1 0.1215 0.1215 0.1184 SF-ALL-Macro STANFORD_KB_SPA_2 ALL 0.1851 0.1919 0.1785 LDC-MAX-ALL-Micro STANFORD_KB_SPA_2 0 0.1895 0.3133 0.2362 LDC-MAX-ALL-Micro STANFORD_KB_SPA_2 1 0.2963 0.0860 0.1333 LDC-MAX-ALL-Micro STANFORD_KB_SPA_2 ALL 0.2000 0.2263 0.2124 LDC-MAX-ALL-Macro STANFORD_KB_SPA_2 0 0.2561 0.2705 0.2506 LDC-MAX-ALL-Macro STANFORD_KB_SPA_2 1 0.1083 0.1083 0.1056 LDC-MAX-ALL-Macro STANFORD_KB_SPA_2 ALL 0.2040 0.2132 0.1994 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 0 0.2380 0.2462 0.2293 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 1 0.1083 0.1083 0.1056 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 ALL 0.1922 0.1975 0.1856 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (EVENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_2 0 0.0000 SF-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 0 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0000 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: SUMMARY: This section provides summary of AP scores Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_2 0 0.0000 SF-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 0 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0000 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro STANFORD_KB_SPA_2 0 0.0000 0.0000 0.0000 SF-ALL-Micro STANFORD_KB_SPA_2 ALL 0.0000 0.0000 0.0000 SF-ALL-Macro STANFORD_KB_SPA_2 0 0.0000 0.0000 0.0000 SF-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro STANFORD_KB_SPA_2 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro STANFORD_KB_SPA_2 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro STANFORD_KB_SPA_2 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 0 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0000 0.0000 0.0000 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. --------------------- Composite KB Evaluation (SENTIMENT-SLOTS only): Scores based on confidence values and Average Precision (AP); k=3 justifications allowed: Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_2 0 0.0000 SF-ALL-Macro STANFORD_KB_SPA_2 1 0.0000 SF-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 0 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 1 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0000 (PRIMARY METRIC) *ALL-Macro AP refer to mean of corresponding AP values. Scores based on confidence values and Average Precision (AP); only k=1 justification allowed: Metric RunID Hop AP SF-ALL-Macro STANFORD_KB_SPA_2 0 0.0000 SF-ALL-Macro STANFORD_KB_SPA_2 1 0.0000 SF-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 0 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 1 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0000 *ALL-Macro AP refer to mean of corresponding AP values. Scores based on P/R/F1; only k=1 justification allowed: Metric RunID Hop Prec Recall F1 SF-ALL-Micro STANFORD_KB_SPA_2 0 0.0000 0.0000 0.0000 SF-ALL-Micro STANFORD_KB_SPA_2 1 0.0000 0.0000 0.0000 SF-ALL-Micro STANFORD_KB_SPA_2 ALL 0.0000 0.0000 0.0000 SF-ALL-Macro STANFORD_KB_SPA_2 0 0.0000 0.0000 0.0000 SF-ALL-Macro STANFORD_KB_SPA_2 1 0.0000 0.0000 0.0000 SF-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro STANFORD_KB_SPA_2 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro STANFORD_KB_SPA_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro STANFORD_KB_SPA_2 ALL 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro STANFORD_KB_SPA_2 0 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro STANFORD_KB_SPA_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 0 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro STANFORD_KB_SPA_2 ALL 0.0000 0.0000 0.0000 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1.