==================================================== TAC KBP 2016 SPANISH SLOT FILLING EVALUATION RESULTS ==================================================== Team ID: UMass_IESL Organization: College of Information and Computer Sciences, University of Massachusetts Amherst ************************************************************* Run ID: UMass_IESL_SF_SPA_1 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro UMass_IESL_SF_SPA_1 0 0.1438 0.2681 0.1872 SF-ALL-Micro UMass_IESL_SF_SPA_1 1 0.0000 0.0000 0.0000 SF-ALL-Micro UMass_IESL_SF_SPA_1 ALL 0.1438 0.1633 0.1529 SF-ALL-Macro UMass_IESL_SF_SPA_1 0 0.1705 0.2635 0.1852 SF-ALL-Macro UMass_IESL_SF_SPA_1 1 0.0000 0.0000 0.0000 SF-ALL-Macro UMass_IESL_SF_SPA_1 ALL 0.1137 0.1757 0.1235 LDC-MAX-ALL-Micro UMass_IESL_SF_SPA_1 0 0.1327 0.2691 0.1777 LDC-MAX-ALL-Micro UMass_IESL_SF_SPA_1 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro UMass_IESL_SF_SPA_1 ALL 0.1327 0.1787 0.1523 LDC-MAX-ALL-Macro UMass_IESL_SF_SPA_1 0 0.1720 0.2587 0.1862 LDC-MAX-ALL-Macro UMass_IESL_SF_SPA_1 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro UMass_IESL_SF_SPA_1 ALL 0.1155 0.1737 0.1250 LDC-MEAN-ALL-Macro UMass_IESL_SF_SPA_1 0 0.1683 0.2551 0.1825 LDC-MEAN-ALL-Macro UMass_IESL_SF_SPA_1 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro UMass_IESL_SF_SPA_1 ALL 0.1130 0.1713 0.1225 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2710 0.8130 0.4065 ************************************************************* Run ID: UMass_IESL_SF_SPA_2 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro UMass_IESL_SF_SPA_2 0 0.1453 0.2349 0.1795 SF-ALL-Micro UMass_IESL_SF_SPA_2 1 0.0000 0.0000 0.0000 SF-ALL-Micro UMass_IESL_SF_SPA_2 ALL 0.1453 0.1431 0.1442 SF-ALL-Macro UMass_IESL_SF_SPA_2 0 0.1747 0.2510 0.1851 SF-ALL-Macro UMass_IESL_SF_SPA_2 1 0.0000 0.0000 0.0000 SF-ALL-Macro UMass_IESL_SF_SPA_2 ALL 0.1165 0.1674 0.1234 LDC-MAX-ALL-Micro UMass_IESL_SF_SPA_2 0 0.1323 0.2369 0.1698 LDC-MAX-ALL-Micro UMass_IESL_SF_SPA_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro UMass_IESL_SF_SPA_2 ALL 0.1323 0.1573 0.1437 LDC-MAX-ALL-Macro UMass_IESL_SF_SPA_2 0 0.1726 0.2398 0.1807 LDC-MAX-ALL-Macro UMass_IESL_SF_SPA_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro UMass_IESL_SF_SPA_2 ALL 0.1159 0.1610 0.1214 LDC-MEAN-ALL-Macro UMass_IESL_SF_SPA_2 0 0.1690 0.2362 0.1771 LDC-MEAN-ALL-Macro UMass_IESL_SF_SPA_2 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro UMass_IESL_SF_SPA_2 ALL 0.1135 0.1586 0.1190 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2730 0.8211 0.4098 ************************************************************* Run ID: UMass_IESL_SF_SPA_3 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro UMass_IESL_SF_SPA_3 0 0.0368 0.3223 0.0661 SF-ALL-Micro UMass_IESL_SF_SPA_3 1 0.0000 0.0000 0.0000 SF-ALL-Micro UMass_IESL_SF_SPA_3 ALL 0.0368 0.1963 0.0620 SF-ALL-Macro UMass_IESL_SF_SPA_3 0 0.1357 0.2413 0.1384 SF-ALL-Macro UMass_IESL_SF_SPA_3 1 0.0000 0.0000 0.0000 SF-ALL-Macro UMass_IESL_SF_SPA_3 ALL 0.0905 0.1609 0.0923 LDC-MAX-ALL-Micro UMass_IESL_SF_SPA_3 0 0.0419 0.3333 0.0745 LDC-MAX-ALL-Micro UMass_IESL_SF_SPA_3 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro UMass_IESL_SF_SPA_3 ALL 0.0419 0.2213 0.0705 LDC-MAX-ALL-Macro UMass_IESL_SF_SPA_3 0 0.1340 0.2407 0.1393 LDC-MAX-ALL-Macro UMass_IESL_SF_SPA_3 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro UMass_IESL_SF_SPA_3 ALL 0.0900 0.1616 0.0935 LDC-MEAN-ALL-Macro UMass_IESL_SF_SPA_3 0 0.1340 0.2398 0.1392 LDC-MEAN-ALL-Macro UMass_IESL_SF_SPA_3 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro UMass_IESL_SF_SPA_3 ALL 0.0900 0.1610 0.0935 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2730 0.8211 0.4098 ************************************************************* Run ID: UMass_IESL_SF_SPA_4 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro UMass_IESL_SF_SPA_4 0 0.1502 0.2831 0.1962 SF-ALL-Micro UMass_IESL_SF_SPA_4 1 0.0000 0.0000 0.0000 SF-ALL-Micro UMass_IESL_SF_SPA_4 ALL 0.1502 0.1725 0.1605 SF-ALL-Macro UMass_IESL_SF_SPA_4 0 0.1760 0.2783 0.1930 SF-ALL-Macro UMass_IESL_SF_SPA_4 1 0.0000 0.0000 0.0000 SF-ALL-Macro UMass_IESL_SF_SPA_4 ALL 0.1173 0.1855 0.1286 LDC-MAX-ALL-Micro UMass_IESL_SF_SPA_4 0 0.1387 0.2851 0.1866 LDC-MAX-ALL-Micro UMass_IESL_SF_SPA_4 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro UMass_IESL_SF_SPA_4 ALL 0.1387 0.1893 0.1601 LDC-MAX-ALL-Macro UMass_IESL_SF_SPA_4 0 0.1806 0.2769 0.1970 LDC-MAX-ALL-Macro UMass_IESL_SF_SPA_4 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro UMass_IESL_SF_SPA_4 ALL 0.1213 0.1859 0.1323 LDC-MEAN-ALL-Macro UMass_IESL_SF_SPA_4 0 0.1769 0.2733 0.1934 LDC-MEAN-ALL-Macro UMass_IESL_SF_SPA_4 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro UMass_IESL_SF_SPA_4 ALL 0.1188 0.1835 0.1298 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2690 0.8049 0.4032 ************************************************************* Run ID: UMass_IESL_SF_SPA_5 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro UMass_IESL_SF_SPA_5 0 0.0367 0.3072 0.0655 SF-ALL-Micro UMass_IESL_SF_SPA_5 1 0.0000 0.0000 0.0000 SF-ALL-Micro UMass_IESL_SF_SPA_5 ALL 0.0367 0.1872 0.0613 SF-ALL-Macro UMass_IESL_SF_SPA_5 0 0.1380 0.2260 0.1317 SF-ALL-Macro UMass_IESL_SF_SPA_5 1 0.0000 0.0000 0.0000 SF-ALL-Macro UMass_IESL_SF_SPA_5 ALL 0.0920 0.1507 0.0878 LDC-MAX-ALL-Micro UMass_IESL_SF_SPA_5 0 0.0427 0.3213 0.0754 LDC-MAX-ALL-Micro UMass_IESL_SF_SPA_5 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro UMass_IESL_SF_SPA_5 ALL 0.0427 0.2133 0.0712 LDC-MAX-ALL-Macro UMass_IESL_SF_SPA_5 0 0.1318 0.2223 0.1298 LDC-MAX-ALL-Macro UMass_IESL_SF_SPA_5 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro UMass_IESL_SF_SPA_5 ALL 0.0885 0.1493 0.0872 LDC-MEAN-ALL-Macro UMass_IESL_SF_SPA_5 0 0.1318 0.2214 0.1298 LDC-MEAN-ALL-Macro UMass_IESL_SF_SPA_5 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro UMass_IESL_SF_SPA_5 ALL 0.0885 0.1487 0.0871 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2730 0.8211 0.4098