==================================================== TAC KBP 2016 ENGLISH SLOT FILLING EVALUATION RESULTS ==================================================== Team ID: NAIST_CL Organization: Computational Linguistics Lab in Nara Institute of Science and Technology ************************************************************* Run ID: NAIST_CL_SF_ENG_1 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: No Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro NAIST_CL_SF_ENG_1 0 0.0937 0.0424 0.0584 SF-ALL-Micro NAIST_CL_SF_ENG_1 1 0.0034 0.1176 0.0066 SF-ALL-Micro NAIST_CL_SF_ENG_1 ALL 0.0057 0.0678 0.0105 SF-ALL-Macro NAIST_CL_SF_ENG_1 0 0.1037 0.0971 0.0991 SF-ALL-Macro NAIST_CL_SF_ENG_1 1 0.0123 0.0406 0.0164 SF-ALL-Macro NAIST_CL_SF_ENG_1 ALL 0.0689 0.0756 0.0677 LDC-MAX-ALL-Micro NAIST_CL_SF_ENG_1 0 0.0868 0.0376 0.0525 LDC-MAX-ALL-Micro NAIST_CL_SF_ENG_1 1 0.0032 0.0974 0.0061 LDC-MAX-ALL-Micro NAIST_CL_SF_ENG_1 ALL 0.0054 0.0576 0.0099 LDC-MAX-ALL-Macro NAIST_CL_SF_ENG_1 0 0.0991 0.0920 0.0941 LDC-MAX-ALL-Macro NAIST_CL_SF_ENG_1 1 0.0135 0.0367 0.0170 LDC-MAX-ALL-Macro NAIST_CL_SF_ENG_1 ALL 0.0655 0.0702 0.0638 LDC-MEAN-ALL-Macro NAIST_CL_SF_ENG_1 0 0.0970 0.0898 0.0920 LDC-MEAN-ALL-Macro NAIST_CL_SF_ENG_1 1 0.0135 0.0367 0.0170 LDC-MEAN-ALL-Macro NAIST_CL_SF_ENG_1 ALL 0.0642 0.0689 0.0625 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.1063 0.2602 0.1509 ************************************************************* Run ID: NAIST_CL_SF_ENG_2 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: No Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro NAIST_CL_SF_ENG_2 0 0.0937 0.0424 0.0584 SF-ALL-Micro NAIST_CL_SF_ENG_2 1 0.0000 0.0000 0.0000 SF-ALL-Micro NAIST_CL_SF_ENG_2 ALL 0.0937 0.0281 0.0433 SF-ALL-Macro NAIST_CL_SF_ENG_2 0 0.1037 0.0971 0.0991 SF-ALL-Macro NAIST_CL_SF_ENG_2 1 0.0000 0.0000 0.0000 SF-ALL-Macro NAIST_CL_SF_ENG_2 ALL 0.0643 0.0602 0.0614 LDC-MAX-ALL-Micro NAIST_CL_SF_ENG_2 0 0.0858 0.0376 0.0523 LDC-MAX-ALL-Micro NAIST_CL_SF_ENG_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro NAIST_CL_SF_ENG_2 ALL 0.0858 0.0250 0.0387 LDC-MAX-ALL-Macro NAIST_CL_SF_ENG_2 0 0.0991 0.0920 0.0941 LDC-MAX-ALL-Macro NAIST_CL_SF_ENG_2 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro NAIST_CL_SF_ENG_2 ALL 0.0602 0.0558 0.0572 LDC-MEAN-ALL-Macro NAIST_CL_SF_ENG_2 0 0.0970 0.0898 0.0920 LDC-MEAN-ALL-Macro NAIST_CL_SF_ENG_2 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro NAIST_CL_SF_ENG_2 ALL 0.0589 0.0545 0.0558 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.1063 0.2602 0.1509 ************************************************************* Run ID: NAIST_CL_SF_ENG_3 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: No Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro NAIST_CL_SF_ENG_3 0 0.0937 0.0424 0.0584 SF-ALL-Micro NAIST_CL_SF_ENG_3 1 0.0000 0.0000 0.0000 SF-ALL-Micro NAIST_CL_SF_ENG_3 ALL 0.0888 0.0281 0.0427 SF-ALL-Macro NAIST_CL_SF_ENG_3 0 0.1037 0.0971 0.0991 SF-ALL-Macro NAIST_CL_SF_ENG_3 1 0.0000 0.0000 0.0000 SF-ALL-Macro NAIST_CL_SF_ENG_3 ALL 0.0643 0.0602 0.0614 LDC-MAX-ALL-Micro NAIST_CL_SF_ENG_3 0 0.0861 0.0376 0.0523 LDC-MAX-ALL-Micro NAIST_CL_SF_ENG_3 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Micro NAIST_CL_SF_ENG_3 ALL 0.0801 0.0250 0.0381 LDC-MAX-ALL-Macro NAIST_CL_SF_ENG_3 0 0.0991 0.0920 0.0941 LDC-MAX-ALL-Macro NAIST_CL_SF_ENG_3 1 0.0000 0.0000 0.0000 LDC-MAX-ALL-Macro NAIST_CL_SF_ENG_3 ALL 0.0602 0.0558 0.0572 LDC-MEAN-ALL-Macro NAIST_CL_SF_ENG_3 0 0.0970 0.0898 0.0920 LDC-MEAN-ALL-Macro NAIST_CL_SF_ENG_3 1 0.0000 0.0000 0.0000 LDC-MEAN-ALL-Macro NAIST_CL_SF_ENG_3 ALL 0.0589 0.0545 0.0558 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.1063 0.2602 0.1509