==================================================== TAC KBP 2016 ENGLISH SLOT FILLING EVALUATION RESULTS ==================================================== Team ID: IRTSX Organization: IRT-SystemX & LIMSI ************************************************************* Run ID: IRTSX_SF_ENG_1 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: No Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro IRTSX_SF_ENG_1 0 0.0105 0.0050 0.0068 SF-ALL-Micro IRTSX_SF_ENG_1 1 0.0000 0.0000 0.0000 SF-ALL-Micro IRTSX_SF_ENG_1 ALL 0.0063 0.0033 0.0043 SF-ALL-Macro IRTSX_SF_ENG_1 0 0.0028 0.0086 0.0042 SF-ALL-Macro IRTSX_SF_ENG_1 1 0.0000 0.0000 0.0000 SF-ALL-Macro IRTSX_SF_ENG_1 ALL 0.0017 0.0054 0.0026 LDC-MAX-ALL-Micro IRTSX_SF_ENG_1 0 0.0175 0.0065 0.0095 LDC-MAX-ALL-Micro IRTSX_SF_ENG_1 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro IRTSX_SF_ENG_1 ALL 0.0133 0.0043 0.0066 LDC-MAX-ALL-Macro IRTSX_SF_ENG_1 0 0.0040 0.0122 0.0060 LDC-MAX-ALL-Macro IRTSX_SF_ENG_1 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro IRTSX_SF_ENG_1 ALL 0.0024 0.0074 0.0036 LDC-MEAN-ALL-Macro IRTSX_SF_ENG_1 0 0.0025 0.0079 0.0038 LDC-MEAN-ALL-Macro IRTSX_SF_ENG_1 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro IRTSX_SF_ENG_1 ALL 0.0015 0.0048 0.0023 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2921 0.9024 0.4413 ************************************************************* Run ID: IRTSX_SF_ENG_2 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: No Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro IRTSX_SF_ENG_2 0 0.0118 0.0050 0.0070 SF-ALL-Micro IRTSX_SF_ENG_2 1 0.0000 0.0000 0.0000 SF-ALL-Micro IRTSX_SF_ENG_2 ALL 0.0061 0.0033 0.0043 SF-ALL-Macro IRTSX_SF_ENG_2 0 0.0032 0.0081 0.0045 SF-ALL-Macro IRTSX_SF_ENG_2 1 0.0000 0.0000 0.0000 SF-ALL-Macro IRTSX_SF_ENG_2 ALL 0.0020 0.0050 0.0028 LDC-MAX-ALL-Micro IRTSX_SF_ENG_2 0 0.0207 0.0065 0.0099 LDC-MAX-ALL-Micro IRTSX_SF_ENG_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro IRTSX_SF_ENG_2 ALL 0.0164 0.0043 0.0069 LDC-MAX-ALL-Macro IRTSX_SF_ENG_2 0 0.0045 0.0115 0.0063 LDC-MAX-ALL-Macro IRTSX_SF_ENG_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro IRTSX_SF_ENG_2 ALL 0.0027 0.0070 0.0038 LDC-MEAN-ALL-Macro IRTSX_SF_ENG_2 0 0.0031 0.0072 0.0042 LDC-MEAN-ALL-Macro IRTSX_SF_ENG_2 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro IRTSX_SF_ENG_2 ALL 0.0019 0.0044 0.0025 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.3013 0.9431 0.4567 ************************************************************* Run ID: IRTSX_SF_ENG_3 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: No Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro IRTSX_SF_ENG_3 0 0.0129 0.0037 0.0058 SF-ALL-Micro IRTSX_SF_ENG_3 1 0.0000 0.0000 0.0000 SF-ALL-Micro IRTSX_SF_ENG_3 ALL 0.0123 0.0025 0.0041 SF-ALL-Macro IRTSX_SF_ENG_3 0 0.0050 0.0051 0.0049 SF-ALL-Macro IRTSX_SF_ENG_3 1 0.0000 0.0000 0.0000 SF-ALL-Macro IRTSX_SF_ENG_3 ALL 0.0031 0.0032 0.0030 LDC-MAX-ALL-Micro IRTSX_SF_ENG_3 0 0.0219 0.0049 0.0080 LDC-MAX-ALL-Micro IRTSX_SF_ENG_3 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro IRTSX_SF_ENG_3 ALL 0.0201 0.0033 0.0056 LDC-MAX-ALL-Macro IRTSX_SF_ENG_3 0 0.0071 0.0072 0.0069 LDC-MAX-ALL-Macro IRTSX_SF_ENG_3 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro IRTSX_SF_ENG_3 ALL 0.0043 0.0044 0.0042 LDC-MEAN-ALL-Macro IRTSX_SF_ENG_3 0 0.0038 0.0043 0.0039 LDC-MEAN-ALL-Macro IRTSX_SF_ENG_3 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro IRTSX_SF_ENG_3 ALL 0.0023 0.0026 0.0024 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.3067 0.9675 0.4658