====================================================================== TAC KBP 2016 EVENT NUGGET DETECTION AND COREFERENCE EVALUATION RESULTS ====================================================================== Team ID: LTI_CMU Organization: Carnegie Mellon University ************************************************************* Run ID: LTI_CMU1 Did the run access the live Web during the evaluation window: No ==========English========== =======Final Mention Detection Results========= Micro Average Macro Average Attributes Prec Rec F1 Prec Rec F1 plain 69.82 39.54 50.49 66.86 38.65 48.99 mention_type 61.69 34.94 44.61 58.46 34.17 43.13 realis_status 45.78 25.93 33.11 40.89 24.52 30.65 mention_type+realis_status 40.19 22.76 29.06 35.21 21.38 26.61 =======Final Mention Coreference Results========= Metric : bcub Score 35.06 Metric : ceafe Score 30.45 Metric : ceafm Score 33.00 * Metric : muc Score 24.60 Metric : blanc Score 18.79 Overall Average CoNLL score 27.23 * Score not included for final CoNLL score. ==========Chinese========== =======Final Mention Detection Results========= Micro Average Macro Average Attributes Prec Rec F1 Prec Rec F1 plain 56.46 39.55 46.52 53.90 38.74 45.08 mention_type 50.72 35.53 41.79 47.89 34.88 40.36 realis_status 42.70 29.92 35.18 40.43 28.78 33.62 mention_type+realis_status 38.91 27.26 32.06 36.39 26.28 30.52 =======Final Mention Coreference Results========= Metric : bcub Score 30.83 Metric : ceafe Score 26.95 Metric : ceafm Score 30.07 * Metric : muc Score 24.07 Metric : blanc Score 17.67 Overall Average CoNLL score 24.88 * Score not included for final CoNLL score. ************************************************************* Run ID: LTI_CMU2 Did the run access the live Web during the evaluation window: No ==========English========== =======Final Mention Detection Results========= Micro Average Macro Average Attributes Prec Rec F1 Prec Rec F1 plain 63.61 30.34 41.09 56.96 27.31 36.92 mention_type 56.33 26.87 36.39 48.42 23.80 31.92 realis_status 43.19 20.60 27.90 37.10 18.09 24.33 mention_type+realis_status 38.59 18.41 24.92 31.68 15.89 21.16 =======Final Mention Coreference Results========= Metric : bcub Score 28.89 Metric : ceafe Score 27.13 Metric : ceafm Score 28.58 * Metric : muc Score 16.85 Metric : blanc Score 15.09 Overall Average CoNLL score 21.99 * Score not included for final CoNLL score. ************************************************************* Run ID: LTI_CMU3 Did the run access the live Web during the evaluation window: No ==========English========== =======Final Mention Detection Results========= Micro Average Macro Average Attributes Prec Rec F1 Prec Rec F1 plain 69.82 39.54 50.49 66.86 38.65 48.99 mention_type 61.69 34.94 44.61 58.46 34.17 43.13 realis_status 45.78 25.93 33.11 40.89 24.52 30.65 mention_type+realis_status 40.19 22.76 29.06 35.21 21.38 26.61 =======Final Mention Coreference Results========= Metric : bcub Score 35.06 Metric : ceafe Score 30.45 Metric : ceafm Score 33.00 * Metric : muc Score 24.60 Metric : blanc Score 18.79 Overall Average CoNLL score 27.23 * Score not included for final CoNLL score. ==========Chinese========== =======Final Mention Detection Results========= Micro Average Macro Average Attributes Prec Rec F1 Prec Rec F1 plain 56.19 35.35 43.40 53.78 35.02 42.42 mention_type 49.70 31.26 38.38 47.02 30.99 37.36 realis_status 43.11 27.12 33.29 41.26 26.58 32.33 mention_type+realis_status 38.54 24.25 29.77 36.52 23.92 28.91 =======Final Mention Coreference Results========= Metric : bcub Score 30.36 Metric : ceafe Score 29.24 Metric : ceafm Score 30.76 * Metric : muc Score 22.99 Metric : blanc Score 16.22 Overall Average CoNLL score 24.70 * Score not included for final CoNLL score.