====================================================================== TAC KBP 2016 EVENT NUGGET DETECTION AND COREFERENCE EVALUATION RESULTS ====================================================================== Team ID: RPI_BLENDER Organization: Rensselaer Polytechnic Institute ************************************************************* Run ID: RPI_BLENDER1 Did the run access the live Web during the evaluation window: No ==========English========== =======Final Mention Detection Results========= Micro Average Macro Average Attributes Prec Rec F1 Prec Rec F1 plain 54.56 22.80 32.16 48.87 22.03 30.37 mention_type 48.17 20.12 28.39 40.93 19.24 26.17 realis_status 41.36 17.28 24.38 38.16 16.69 23.22 mention_type+realis_status 36.52 15.26 21.52 32.06 14.50 19.97 =======Final Mention Coreference Results========= Metric : bcub Score 20.96 Metric : ceafe Score 16.14 Metric : ceafm Score 19.65 * Metric : muc Score 17.32 Metric : blanc Score 10.67 Overall Average CoNLL score 16.27 * Score not included for final CoNLL score. ==========Chinese========== =======Final Mention Detection Results========= Micro Average Macro Average Attributes Prec Rec F1 Prec Rec F1 plain 59.91 14.44 23.27 37.87 12.84 19.18 mention_type 57.00 13.74 22.14 35.81 12.29 18.30 realis_status 48.71 11.74 18.92 28.00 10.05 14.79 mention_type+realis_status 47.23 11.38 18.35 27.09 9.79 14.38 =======Final Mention Coreference Results========= Metric : bcub Score 17.44 Metric : ceafe Score 11.93 Metric : ceafm Score 15.74 * Metric : muc Score 16.61 Metric : blanc Score 10.13 Overall Average CoNLL score 14.03 * Score not included for final CoNLL score. ==========Spanish========== =======Final Mention Detection Results========= Micro Average Macro Average Attributes Prec Rec F1 Prec Rec F1 plain 50.00 36.94 42.49 48.94 35.31 41.03 mention_type 36.56 27.01 31.06 34.31 25.66 29.36 realis_status 40.23 29.72 34.18 39.03 27.94 32.57 mention_type+realis_status 28.90 21.35 24.56 26.57 19.89 22.75 =======Final Mention Coreference Results========= Metric : bcub Score 22.05 Metric : ceafe Score 18.56 Metric : ceafm Score 23.47 * Metric : muc Score 19.04 Metric : blanc Score 12.43 Overall Average CoNLL score 18.02 * Score not included for final CoNLL score. ************************************************************* Run ID: RPI_BLENDER2 Did the run access the live Web during the evaluation window: No ==========English========== =======Final Mention Detection Results========= Micro Average Macro Average Attributes Prec Rec F1 Prec Rec F1 plain 54.48 22.75 32.09 48.51 21.96 30.23 mention_type 48.19 20.12 28.39 40.74 19.24 26.14 realis_status 41.27 17.23 24.31 37.74 16.62 23.08 mention_type+realis_status 36.54 15.26 21.53 31.66 14.48 19.87 =======Final Mention Coreference Results========= Metric : bcub Score 20.98 Metric : ceafe Score 16.18 Metric : ceafm Score 19.59 * Metric : muc Score 17.28 Metric : blanc Score 10.63 Overall Average CoNLL score 16.27 * Score not included for final CoNLL score. ==========Chinese========== =======Final Mention Detection Results========= Micro Average Macro Average Attributes Prec Rec F1 Prec Rec F1 plain 60.65 14.40 23.28 38.14 12.80 19.17 mention_type 57.69 13.70 22.14 36.04 12.25 18.29 realis_status 49.44 11.74 18.98 28.28 10.05 14.83 mention_type+realis_status 47.94 11.38 18.40 27.33 9.78 14.41 =======Final Mention Coreference Results========= Metric : bcub Score 17.46 Metric : ceafe Score 11.97 Metric : ceafm Score 15.78 * Metric : muc Score 16.51 Metric : blanc Score 10.23 Overall Average CoNLL score 14.04 * Score not included for final CoNLL score. ==========Spanish========== =======Final Mention Detection Results========= Micro Average Macro Average Attributes Prec Rec F1 Prec Rec F1 plain 49.92 36.81 42.37 48.12 35.09 40.58 mention_type 37.02 27.30 31.43 34.08 25.77 29.35 realis_status 40.07 29.55 34.02 38.15 27.69 32.09 mention_type+realis_status 29.29 21.60 24.87 26.31 19.95 22.69 =======Final Mention Coreference Results========= Metric : bcub Score 22.19 Metric : ceafe Score 18.47 Metric : ceafm Score 23.58 * Metric : muc Score 19.13 Metric : blanc Score 12.75 Overall Average CoNLL score 18.13 * Score not included for final CoNLL score. ************************************************************* Run ID: RPI_BLENDER3 Did the run access the live Web during the evaluation window: No ==========English========== =======Final Mention Detection Results========= Micro Average Macro Average Attributes Prec Rec F1 Prec Rec F1 plain 54.48 22.75 32.09 48.51 21.96 30.23 mention_type 48.19 20.12 28.39 40.74 19.24 26.14 realis_status 41.27 17.23 24.31 37.74 16.62 23.08 mention_type+realis_status 36.54 15.26 21.53 31.66 14.48 19.87 =======Final Mention Coreference Results========= Metric : bcub Score 20.98 Metric : ceafe Score 16.18 Metric : ceafm Score 19.59 * Metric : muc Score 17.28 Metric : blanc Score 10.63 Overall Average CoNLL score 16.27 * Score not included for final CoNLL score. ==========Chinese========== =======Final Mention Detection Results========= Micro Average Macro Average Attributes Prec Rec F1 Prec Rec F1 plain 59.91 14.44 23.27 37.87 12.84 19.18 mention_type 57.00 13.74 22.14 35.81 12.29 18.30 realis_status 48.71 11.74 18.92 28.00 10.05 14.79 mention_type+realis_status 47.23 11.38 18.35 27.09 9.79 14.38 =======Final Mention Coreference Results========= Metric : bcub Score 17.44 Metric : ceafe Score 11.93 Metric : ceafm Score 15.74 * Metric : muc Score 16.61 Metric : blanc Score 10.13 Overall Average CoNLL score 14.03 * Score not included for final CoNLL score. ==========Spanish========== =======Final Mention Detection Results========= Micro Average Macro Average Attributes Prec Rec F1 Prec Rec F1 plain 50.00 36.94 42.49 48.94 35.31 41.03 mention_type 36.56 27.01 31.06 34.31 25.66 29.36 realis_status 40.23 29.72 34.18 39.03 27.94 32.57 mention_type+realis_status 28.90 21.35 24.56 26.57 19.89 22.75 =======Final Mention Coreference Results========= Metric : bcub Score 22.05 Metric : ceafe Score 18.56 Metric : ceafm Score 23.47 * Metric : muc Score 19.04 Metric : blanc Score 12.43 Overall Average CoNLL score 18.02 * Score not included for final CoNLL score.