==================================================== TAC KBP 2016 ENGLISH SLOT FILLING EVALUATION RESULTS ==================================================== Team ID: RPI_BLENDER Organization: Rensselaer Polytechnic Institute ************************************************************* Run ID: RPI_BLENDER_SF_ENG_1 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro RPI_BLENDER_SF_ENG_1 0 0.3519 0.1186 0.1774 SF-ALL-Micro RPI_BLENDER_SF_ENG_1 1 0.3514 0.0637 0.1079 SF-ALL-Micro RPI_BLENDER_SF_ENG_1 ALL 0.3517 0.1001 0.1558 SF-ALL-Macro RPI_BLENDER_SF_ENG_1 0 0.1845 0.2030 0.1845 SF-ALL-Macro RPI_BLENDER_SF_ENG_1 1 0.0546 0.0614 0.0522 SF-ALL-Macro RPI_BLENDER_SF_ENG_1 ALL 0.1351 0.1492 0.1342 LDC-MAX-ALL-Micro RPI_BLENDER_SF_ENG_1 0 0.3582 0.1176 0.1771 LDC-MAX-ALL-Micro RPI_BLENDER_SF_ENG_1 1 0.2963 0.0519 0.0884 LDC-MAX-ALL-Micro RPI_BLENDER_SF_ENG_1 ALL 0.3451 0.0957 0.1498 LDC-MAX-ALL-Macro RPI_BLENDER_SF_ENG_1 0 0.1956 0.2056 0.1913 LDC-MAX-ALL-Macro RPI_BLENDER_SF_ENG_1 1 0.0493 0.0589 0.0494 LDC-MAX-ALL-Macro RPI_BLENDER_SF_ENG_1 ALL 0.1382 0.1480 0.1356 LDC-MEAN-ALL-Macro RPI_BLENDER_SF_ENG_1 0 0.1726 0.1887 0.1713 LDC-MEAN-ALL-Macro RPI_BLENDER_SF_ENG_1 1 0.0460 0.0556 0.0461 LDC-MEAN-ALL-Macro RPI_BLENDER_SF_ENG_1 ALL 0.1229 0.1364 0.1221 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2846 0.8699 0.4289 ************************************************************* Run ID: RPI_BLENDER_SF_ENG_2 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro RPI_BLENDER_SF_ENG_2 0 0.3441 0.1061 0.1622 SF-ALL-Micro RPI_BLENDER_SF_ENG_2 1 0.3429 0.0588 0.1004 SF-ALL-Micro RPI_BLENDER_SF_ENG_2 ALL 0.3438 0.0902 0.1429 SF-ALL-Macro RPI_BLENDER_SF_ENG_2 0 0.1581 0.1791 0.1600 SF-ALL-Macro RPI_BLENDER_SF_ENG_2 1 0.0570 0.0605 0.0513 SF-ALL-Macro RPI_BLENDER_SF_ENG_2 ALL 0.1197 0.1340 0.1187 LDC-MAX-ALL-Micro RPI_BLENDER_SF_ENG_2 0 0.3523 0.1013 0.1574 LDC-MAX-ALL-Micro RPI_BLENDER_SF_ENG_2 1 0.3061 0.0487 0.0840 LDC-MAX-ALL-Micro RPI_BLENDER_SF_ENG_2 ALL 0.3422 0.0837 0.1345 LDC-MAX-ALL-Macro RPI_BLENDER_SF_ENG_2 0 0.1582 0.1719 0.1567 LDC-MAX-ALL-Macro RPI_BLENDER_SF_ENG_2 1 0.0510 0.0583 0.0488 LDC-MAX-ALL-Macro RPI_BLENDER_SF_ENG_2 ALL 0.1161 0.1273 0.1144 LDC-MEAN-ALL-Macro RPI_BLENDER_SF_ENG_2 0 0.1449 0.1632 0.1454 LDC-MEAN-ALL-Macro RPI_BLENDER_SF_ENG_2 1 0.0477 0.0550 0.0455 LDC-MEAN-ALL-Macro RPI_BLENDER_SF_ENG_2 ALL 0.1067 0.1207 0.1062 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2902 0.8943 0.4382 ************************************************************* Run ID: RPI_BLENDER_SF_ENG_3 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro RPI_BLENDER_SF_ENG_3 0 0.3232 0.1061 0.1598 SF-ALL-Micro RPI_BLENDER_SF_ENG_3 1 0.3243 0.0588 0.0996 SF-ALL-Micro RPI_BLENDER_SF_ENG_3 ALL 0.3234 0.0902 0.1410 SF-ALL-Macro RPI_BLENDER_SF_ENG_3 0 0.1566 0.1791 0.1590 SF-ALL-Macro RPI_BLENDER_SF_ENG_3 1 0.0559 0.0605 0.0505 SF-ALL-Macro RPI_BLENDER_SF_ENG_3 ALL 0.1183 0.1340 0.1178 LDC-MAX-ALL-Micro RPI_BLENDER_SF_ENG_3 0 0.3196 0.1013 0.1538 LDC-MAX-ALL-Micro RPI_BLENDER_SF_ENG_3 1 0.2778 0.0487 0.0829 LDC-MAX-ALL-Micro RPI_BLENDER_SF_ENG_3 ALL 0.3105 0.0837 0.1318 LDC-MAX-ALL-Macro RPI_BLENDER_SF_ENG_3 0 0.1565 0.1719 0.1556 LDC-MAX-ALL-Macro RPI_BLENDER_SF_ENG_3 1 0.0502 0.0583 0.0483 LDC-MAX-ALL-Macro RPI_BLENDER_SF_ENG_3 ALL 0.1148 0.1273 0.1135 LDC-MEAN-ALL-Macro RPI_BLENDER_SF_ENG_3 0 0.1436 0.1632 0.1446 LDC-MEAN-ALL-Macro RPI_BLENDER_SF_ENG_3 1 0.0469 0.0550 0.0450 LDC-MEAN-ALL-Macro RPI_BLENDER_SF_ENG_3 ALL 0.1056 0.1207 0.1055 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2884 0.8862 0.4352