==================================================== TAC KBP 2016 ENGLISH SLOT FILLING EVALUATION RESULTS ==================================================== Team ID: UNIST_SAIL Organization: Ulsan National Institute of Science of Technology ************************************************************* Run ID: UNIST_SAIL_SF_ENG_1 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro UNIST_SAIL_SF_ENG_1 0 0.0870 0.0275 0.0417 SF-ALL-Micro UNIST_SAIL_SF_ENG_1 1 0.0000 0.0000 0.0000 SF-ALL-Micro UNIST_SAIL_SF_ENG_1 ALL 0.0087 0.0182 0.0118 SF-ALL-Macro UNIST_SAIL_SF_ENG_1 0 0.0224 0.0336 0.0240 SF-ALL-Macro UNIST_SAIL_SF_ENG_1 1 0.0000 0.0000 0.0000 SF-ALL-Macro UNIST_SAIL_SF_ENG_1 ALL 0.0139 0.0208 0.0149 LDC-MAX-ALL-Micro UNIST_SAIL_SF_ENG_1 0 0.0810 0.0278 0.0414 LDC-MAX-ALL-Micro UNIST_SAIL_SF_ENG_1 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro UNIST_SAIL_SF_ENG_1 ALL 0.0127 0.0185 0.0150 LDC-MAX-ALL-Macro UNIST_SAIL_SF_ENG_1 0 0.0231 0.0375 0.0249 LDC-MAX-ALL-Macro UNIST_SAIL_SF_ENG_1 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro UNIST_SAIL_SF_ENG_1 ALL 0.0140 0.0228 0.0151 LDC-MEAN-ALL-Macro UNIST_SAIL_SF_ENG_1 0 0.0213 0.0342 0.0227 LDC-MEAN-ALL-Macro UNIST_SAIL_SF_ENG_1 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro UNIST_SAIL_SF_ENG_1 ALL 0.0129 0.0208 0.0138 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2921 0.9024 0.4413 ************************************************************* Run ID: UNIST_SAIL_SF_ENG_2 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro UNIST_SAIL_SF_ENG_2 0 0.0751 0.0350 0.0477 SF-ALL-Micro UNIST_SAIL_SF_ENG_2 1 0.0000 0.0000 0.0000 SF-ALL-Micro UNIST_SAIL_SF_ENG_2 ALL 0.0751 0.0232 0.0354 SF-ALL-Macro UNIST_SAIL_SF_ENG_2 0 0.0217 0.0434 0.0259 SF-ALL-Macro UNIST_SAIL_SF_ENG_2 1 0.0000 0.0000 0.0000 SF-ALL-Macro UNIST_SAIL_SF_ENG_2 ALL 0.0134 0.0269 0.0160 LDC-MAX-ALL-Micro UNIST_SAIL_SF_ENG_2 0 0.0700 0.0343 0.0461 LDC-MAX-ALL-Micro UNIST_SAIL_SF_ENG_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro UNIST_SAIL_SF_ENG_2 ALL 0.0700 0.0228 0.0344 LDC-MAX-ALL-Macro UNIST_SAIL_SF_ENG_2 0 0.0233 0.0477 0.0277 LDC-MAX-ALL-Macro UNIST_SAIL_SF_ENG_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro UNIST_SAIL_SF_ENG_2 ALL 0.0141 0.0289 0.0169 LDC-MEAN-ALL-Macro UNIST_SAIL_SF_ENG_2 0 0.0198 0.0423 0.0239 LDC-MEAN-ALL-Macro UNIST_SAIL_SF_ENG_2 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro UNIST_SAIL_SF_ENG_2 ALL 0.0120 0.0257 0.0145 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2865 0.8780 0.4320 ************************************************************* Run ID: UNIST_SAIL_SF_ENG_3 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro UNIST_SAIL_SF_ENG_3 0 0.0958 0.0375 0.0539 SF-ALL-Micro UNIST_SAIL_SF_ENG_3 1 0.0000 0.0000 0.0000 SF-ALL-Micro UNIST_SAIL_SF_ENG_3 ALL 0.0958 0.0248 0.0394 SF-ALL-Macro UNIST_SAIL_SF_ENG_3 0 0.0274 0.0416 0.0287 SF-ALL-Macro UNIST_SAIL_SF_ENG_3 1 0.0000 0.0000 0.0000 SF-ALL-Macro UNIST_SAIL_SF_ENG_3 ALL 0.0170 0.0258 0.0178 LDC-MAX-ALL-Micro UNIST_SAIL_SF_ENG_3 0 0.0866 0.0359 0.0508 LDC-MAX-ALL-Micro UNIST_SAIL_SF_ENG_3 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro UNIST_SAIL_SF_ENG_3 ALL 0.0866 0.0239 0.0375 LDC-MAX-ALL-Macro UNIST_SAIL_SF_ENG_3 0 0.0285 0.0445 0.0291 LDC-MAX-ALL-Macro UNIST_SAIL_SF_ENG_3 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro UNIST_SAIL_SF_ENG_3 ALL 0.0173 0.0270 0.0177 LDC-MEAN-ALL-Macro UNIST_SAIL_SF_ENG_3 0 0.0250 0.0406 0.0264 LDC-MEAN-ALL-Macro UNIST_SAIL_SF_ENG_3 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro UNIST_SAIL_SF_ENG_3 ALL 0.0152 0.0246 0.0160 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2921 0.9024 0.4413 ************************************************************* Run ID: UNIST_SAIL_SF_ENG_4 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro UNIST_SAIL_SF_ENG_4 0 0.0792 0.0624 0.0698 SF-ALL-Micro UNIST_SAIL_SF_ENG_4 1 0.0000 0.0000 0.0000 SF-ALL-Micro UNIST_SAIL_SF_ENG_4 ALL 0.0792 0.0414 0.0543 SF-ALL-Macro UNIST_SAIL_SF_ENG_4 0 0.0415 0.0546 0.0406 SF-ALL-Macro UNIST_SAIL_SF_ENG_4 1 0.0000 0.0000 0.0000 SF-ALL-Macro UNIST_SAIL_SF_ENG_4 ALL 0.0257 0.0338 0.0252 LDC-MAX-ALL-Micro UNIST_SAIL_SF_ENG_4 0 0.0884 0.0686 0.0773 LDC-MAX-ALL-Micro UNIST_SAIL_SF_ENG_4 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro UNIST_SAIL_SF_ENG_4 ALL 0.0884 0.0457 0.0602 LDC-MAX-ALL-Macro UNIST_SAIL_SF_ENG_4 0 0.0484 0.0628 0.0458 LDC-MAX-ALL-Macro UNIST_SAIL_SF_ENG_4 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro UNIST_SAIL_SF_ENG_4 ALL 0.0294 0.0381 0.0278 LDC-MEAN-ALL-Macro UNIST_SAIL_SF_ENG_4 0 0.0441 0.0581 0.0424 LDC-MEAN-ALL-Macro UNIST_SAIL_SF_ENG_4 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro UNIST_SAIL_SF_ENG_4 ALL 0.0268 0.0353 0.0258 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2507 0.7317 0.3734