==================================================== TAC KBP 2017 BELIEF AND SENTIMENT EVALUATION RESULTS ==================================================== Team ID: ict_spring Organization: Institute of Computing Technology，Chinese Academy ************************************************************* Run ID: ict_spring_sys1 Did the run access the live Web during the evaluation window: No Scores under each evaluation condition: language (cmn, eng, spa), genre (df, nw), input ere (gold-ere, predicted-ere). Scores were computed using lenient setting (full credit if and only if at least one correct provenance is returned). Condition Belief Sentiment All Prec Recall F1 Prec Recall F1 Prec Recall F1 cmn-df-gold-ere 0.8081 0.8368 0.8222 0.0873 0.2865 0.1338 0.5548 0.7565 0.6402 cmn-df-pred-ere 0.0080 0.0018 0.0029 0.0337 0.0329 0.0333 0.0194 0.0066 0.0099 cmn-nw-gold-ere 0.6387 0.3873 0.4822 0.0148 0.1815 0.0273 0.3236 0.3774 0.3484 cmn-nw-pred-ere 0.0095 0.0016 0.0028 0.0072 0.0101 0.0084 0.0089 0.0020 0.0033 eng-df-gold-ere 0.7251 0.8431 0.7797 0.1221 0.5191 0.1977 0.3945 0.7624 0.5199 eng-df-pred-ere 0.0265 0.0027 0.0049 0.0342 0.0336 0.0339 0.0323 0.0105 0.0158 eng-nw-gold-ere 0.7378 0.5072 0.6012 0.0461 0.1968 0.0747 0.4053 0.4669 0.4339 eng-nw-pred-ere 0.0267 0.0042 0.0072 0.0240 0.0165 0.0195 0.0255 0.0059 0.0096 spa-df-gold-ere 0.7990 0.7074 0.7504 0.1285 0.3501 0.1880 0.4522 0.6151 0.5212 spa-df-pred-ere 0.0392 0.0017 0.0033 0.0431 0.0198 0.0272 0.0423 0.0065 0.0112 spa-nw-gold-ere 0.6498 0.4547 0.5350 0.0363 0.2069 0.0618 0.2977 0.4195 0.3483 spa-nw-pred-ere 0.0354 0.0032 0.0059 0.0227 0.0205 0.0215 0.0274 0.0058 0.0095 ************************************************************* Run ID: ict_spring_sys2 Did the run access the live Web during the evaluation window: No Scores under each evaluation condition: language (cmn, eng, spa), genre (df, nw), input ere (gold-ere, predicted-ere). Scores were computed using lenient setting (full credit if and only if at least one correct provenance is returned). Condition Belief Sentiment All Prec Recall F1 Prec Recall F1 Prec Recall F1 cmn-df-gold-ere 0.8080 0.8348 0.8212 0.0590 0.2139 0.0925 0.5272 0.7441 0.6172 cmn-nw-gold-ere 0.6387 0.3871 0.4820 0.0079 0.0521 0.0137 0.4147 0.3710 0.3917 eng-df-gold-ere 0.7354 0.8011 0.7669 0.1096 0.3152 0.1626 0.4431 0.6800 0.5365 eng-nw-gold-ere 0.7386 0.5022 0.5979 0.0460 0.1453 0.0698 0.4550 0.4558 0.4554 spa-df-gold-ere 0.8131 0.6223 0.7050 0.0908 0.3323 0.1427 0.3619 0.5474 0.4358 spa-nw-gold-ere 0.6493 0.4560 0.5357 0.0297 0.1603 0.0501 0.3024 0.4140 0.3495 ************************************************************* Run ID: ict_spring_sys3 Did the run access the live Web during the evaluation window: No Scores under each evaluation condition: language (cmn, eng, spa), genre (df, nw), input ere (gold-ere, predicted-ere). Scores were computed using lenient setting (full credit if and only if at least one correct provenance is returned). Condition Belief Sentiment All Prec Recall F1 Prec Recall F1 Prec Recall F1 cmn-df-gold-ere 0.8081 0.8368 0.8222 0.0487 0.7135 0.0911 0.2707 0.8188 0.4069 cmn-df-pred-ere 0.0074 0.0018 0.0029 0.0184 0.0697 0.0291 0.0156 0.0123 0.0137 cmn-nw-gold-ere 0.6387 0.3873 0.4822 0.0137 0.3140 0.0262 0.2284 0.3837 0.2864 cmn-nw-pred-ere 0.0094 0.0016 0.0028 0.0039 0.0327 0.0070 0.0056 0.0030 0.0039 eng-df-gold-ere 0.7256 0.8521 0.7838 0.0918 0.7084 0.1625 0.2910 0.8163 0.4291 eng-df-pred-ere 0.0229 0.0027 0.0049 0.0302 0.0577 0.0396 0.0290 0.0165 0.0211 eng-nw-gold-ere 0.7373 0.5091 0.6023 0.0405 0.3377 0.0724 0.2894 0.4869 0.3630 eng-nw-pred-ere 0.0265 0.0042 0.0072 0.0175 0.0463 0.0254 0.0199 0.0101 0.0134 spa-df-gold-ere 0.7893 0.7638 0.7764 0.0970 0.6580 0.1691 0.2983 0.7365 0.4246 spa-df-pred-ere 0.0300 0.0017 0.0033 0.0493 0.0719 0.0585 0.0473 0.0200 0.0281 spa-nw-gold-ere 0.6511 0.4612 0.5399 0.0350 0.2834 0.0623 0.2481 0.4360 0.3162 spa-nw-pred-ere 0.0349 0.0032 0.0059 0.0219 0.0523 0.0309 0.0243 0.0104 0.0146