==================================================== TAC KBP 2016 ENGLISH SLOT FILLING EVALUATION RESULTS ==================================================== Team ID: SoochowNLP Organization: Soochow University Natural Language Processing Lab ************************************************************* Run ID: SoochowNLP_SF_ENG_1 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro SoochowNLP_SF_ENG_1 0 0.0424 0.0449 0.0436 SF-ALL-Micro SoochowNLP_SF_ENG_1 1 0.0000 0.0000 0.0000 SF-ALL-Micro SoochowNLP_SF_ENG_1 ALL 0.0424 0.0298 0.0350 SF-ALL-Macro SoochowNLP_SF_ENG_1 0 0.0181 0.0593 0.0234 SF-ALL-Macro SoochowNLP_SF_ENG_1 1 0.0000 0.0000 0.0000 SF-ALL-Macro SoochowNLP_SF_ENG_1 ALL 0.0112 0.0368 0.0145 LDC-MAX-ALL-Micro SoochowNLP_SF_ENG_1 0 0.0474 0.0474 0.0474 LDC-MAX-ALL-Micro SoochowNLP_SF_ENG_1 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro SoochowNLP_SF_ENG_1 ALL 0.0474 0.0315 0.0379 LDC-MAX-ALL-Macro SoochowNLP_SF_ENG_1 0 0.0240 0.0674 0.0303 LDC-MAX-ALL-Macro SoochowNLP_SF_ENG_1 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro SoochowNLP_SF_ENG_1 ALL 0.0146 0.0409 0.0184 LDC-MEAN-ALL-Macro SoochowNLP_SF_ENG_1 0 0.0174 0.0587 0.0230 LDC-MEAN-ALL-Macro SoochowNLP_SF_ENG_1 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro SoochowNLP_SF_ENG_1 ALL 0.0106 0.0357 0.0140 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2788 0.8455 0.4193 ************************************************************* Run ID: SoochowNLP_SF_ENG_2 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro SoochowNLP_SF_ENG_2 0 0.0587 0.0262 0.0362 SF-ALL-Micro SoochowNLP_SF_ENG_2 1 0.0000 0.0000 0.0000 SF-ALL-Micro SoochowNLP_SF_ENG_2 ALL 0.0587 0.0174 0.0268 SF-ALL-Macro SoochowNLP_SF_ENG_2 0 0.0208 0.0357 0.0202 SF-ALL-Macro SoochowNLP_SF_ENG_2 1 0.0000 0.0000 0.0000 SF-ALL-Macro SoochowNLP_SF_ENG_2 ALL 0.0129 0.0221 0.0125 LDC-MAX-ALL-Micro SoochowNLP_SF_ENG_2 0 0.0690 0.0327 0.0443 LDC-MAX-ALL-Micro SoochowNLP_SF_ENG_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro SoochowNLP_SF_ENG_2 ALL 0.0690 0.0217 0.0331 LDC-MAX-ALL-Macro SoochowNLP_SF_ENG_2 0 0.0283 0.0462 0.0268 LDC-MAX-ALL-Macro SoochowNLP_SF_ENG_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro SoochowNLP_SF_ENG_2 ALL 0.0172 0.0280 0.0163 LDC-MEAN-ALL-Macro SoochowNLP_SF_ENG_2 0 0.0194 0.0368 0.0192 LDC-MEAN-ALL-Macro SoochowNLP_SF_ENG_2 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro SoochowNLP_SF_ENG_2 ALL 0.0118 0.0224 0.0117 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2921 0.9024 0.4413 ************************************************************* Run ID: SoochowNLP_SF_ENG_3 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro SoochowNLP_SF_ENG_3 0 0.0587 0.0262 0.0362 SF-ALL-Micro SoochowNLP_SF_ENG_3 1 0.0007 0.0025 0.0011 SF-ALL-Micro SoochowNLP_SF_ENG_3 ALL 0.0129 0.0182 0.0151 SF-ALL-Macro SoochowNLP_SF_ENG_3 0 0.0208 0.0357 0.0202 SF-ALL-Macro SoochowNLP_SF_ENG_3 1 0.0008 0.0050 0.0014 SF-ALL-Macro SoochowNLP_SF_ENG_3 ALL 0.0132 0.0240 0.0130 LDC-MAX-ALL-Micro SoochowNLP_SF_ENG_3 0 0.0673 0.0327 0.0440 LDC-MAX-ALL-Micro SoochowNLP_SF_ENG_3 1 0.0010 0.0032 0.0015 LDC-MAX-ALL-Micro SoochowNLP_SF_ENG_3 ALL 0.0157 0.0228 0.0186 LDC-MAX-ALL-Macro SoochowNLP_SF_ENG_3 0 0.0283 0.0462 0.0268 LDC-MAX-ALL-Macro SoochowNLP_SF_ENG_3 1 0.0011 0.0067 0.0019 LDC-MAX-ALL-Macro SoochowNLP_SF_ENG_3 ALL 0.0177 0.0307 0.0170 LDC-MEAN-ALL-Macro SoochowNLP_SF_ENG_3 0 0.0194 0.0368 0.0192 LDC-MEAN-ALL-Macro SoochowNLP_SF_ENG_3 1 0.0011 0.0067 0.0019 LDC-MEAN-ALL-Macro SoochowNLP_SF_ENG_3 ALL 0.0122 0.0250 0.0124 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2921 0.9024 0.4413 ************************************************************* Run ID: SoochowNLP_SF_ENG_4 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: No Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro SoochowNLP_SF_ENG_4 0 0.0587 0.0262 0.0362 SF-ALL-Micro SoochowNLP_SF_ENG_4 1 0.0075 0.0025 0.0037 SF-ALL-Micro SoochowNLP_SF_ENG_4 ALL 0.0448 0.0182 0.0259 SF-ALL-Macro SoochowNLP_SF_ENG_4 0 0.0208 0.0357 0.0202 SF-ALL-Macro SoochowNLP_SF_ENG_4 1 0.0025 0.0050 0.0033 SF-ALL-Macro SoochowNLP_SF_ENG_4 ALL 0.0139 0.0240 0.0138 LDC-MAX-ALL-Micro SoochowNLP_SF_ENG_4 0 0.0669 0.0327 0.0439 LDC-MAX-ALL-Micro SoochowNLP_SF_ENG_4 1 0.0086 0.0032 0.0047 LDC-MAX-ALL-Micro SoochowNLP_SF_ENG_4 ALL 0.0506 0.0228 0.0315 LDC-MAX-ALL-Macro SoochowNLP_SF_ENG_4 0 0.0283 0.0462 0.0268 LDC-MAX-ALL-Macro SoochowNLP_SF_ENG_4 1 0.0033 0.0067 0.0044 LDC-MAX-ALL-Macro SoochowNLP_SF_ENG_4 ALL 0.0185 0.0307 0.0180 LDC-MEAN-ALL-Macro SoochowNLP_SF_ENG_4 0 0.0194 0.0368 0.0192 LDC-MEAN-ALL-Macro SoochowNLP_SF_ENG_4 1 0.0033 0.0067 0.0044 LDC-MEAN-ALL-Macro SoochowNLP_SF_ENG_4 ALL 0.0131 0.0250 0.0134 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2921 0.9024 0.4413 ************************************************************* Run ID: SoochowNLP_SF_ENG_5 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro SoochowNLP_SF_ENG_5 0 0.0587 0.0262 0.0362 SF-ALL-Micro SoochowNLP_SF_ENG_5 1 0.0053 0.0025 0.0034 SF-ALL-Micro SoochowNLP_SF_ENG_5 ALL 0.0403 0.0182 0.0251 SF-ALL-Macro SoochowNLP_SF_ENG_5 0 0.0208 0.0357 0.0202 SF-ALL-Macro SoochowNLP_SF_ENG_5 1 0.0025 0.0050 0.0033 SF-ALL-Macro SoochowNLP_SF_ENG_5 ALL 0.0139 0.0240 0.0138 LDC-MAX-ALL-Micro SoochowNLP_SF_ENG_5 0 0.0667 0.0327 0.0439 LDC-MAX-ALL-Micro SoochowNLP_SF_ENG_5 1 0.0069 0.0032 0.0044 LDC-MAX-ALL-Micro SoochowNLP_SF_ENG_5 ALL 0.0473 0.0228 0.0308 LDC-MAX-ALL-Macro SoochowNLP_SF_ENG_5 0 0.0283 0.0462 0.0268 LDC-MAX-ALL-Macro SoochowNLP_SF_ENG_5 1 0.0033 0.0067 0.0044 LDC-MAX-ALL-Macro SoochowNLP_SF_ENG_5 ALL 0.0185 0.0307 0.0180 LDC-MEAN-ALL-Macro SoochowNLP_SF_ENG_5 0 0.0194 0.0368 0.0192 LDC-MEAN-ALL-Macro SoochowNLP_SF_ENG_5 1 0.0033 0.0067 0.0044 LDC-MEAN-ALL-Macro SoochowNLP_SF_ENG_5 ALL 0.0131 0.0250 0.0134 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2921 0.9024 0.4413