==================================================== TAC KBP 2017 BELIEF AND SENTIMENT EVALUATION RESULTS ==================================================== Team ID: Columbia_GW Organization: Columbia University ************************************************************* Run ID: Columbia_GW_sys1 Did the run access the live Web during the evaluation window: No Scores under each evaluation condition: language (cmn, eng, spa), genre (df, nw), input ere (gold-ere, predicted-ere). Scores were computed using lenient setting (full credit if and only if at least one correct provenance is returned). Condition Belief Sentiment All Prec Recall F1 Prec Recall F1 Prec Recall F1 cmn-df-gold-ere 0.8156 0.6941 0.7499 0.1944 0.0185 0.0338 0.8039 0.5955 0.6842 cmn-df-pred-ere 0.0111 0.0022 0.0037 0.0000 0.0000 0.0000 0.0110 0.0019 0.0032 cmn-nw-gold-ere 0.8030 0.4789 0.6000 0.2727 0.0134 0.0255 0.8008 0.4566 0.5815 cmn-nw-pred-ere 0.0104 0.0018 0.0031 0.0000 0.0000 0.0000 0.0104 0.0017 0.0029 eng-df-gold-ere 0.7886 0.7440 0.7657 0.1946 0.3515 0.2505 0.5577 0.6462 0.5987 eng-df-pred-ere 0.0323 0.0033 0.0060 0.0974 0.0652 0.0781 0.0771 0.0188 0.0302 eng-nw-gold-ere 0.7782 0.5210 0.6241 0.0670 0.2818 0.1083 0.4341 0.4899 0.4603 eng-nw-pred-ere 0.0286 0.0043 0.0075 0.0217 0.0312 0.0256 0.0244 0.0080 0.0121 spa-df-gold-ere 0.6530 0.7064 0.6787 0.1257 0.5803 0.2066 0.3378 0.6739 0.4500 spa-df-pred-ere 0.0000 0.0000 0.0000 0.0657 0.0765 0.0707 0.0581 0.0199 0.0297 spa-nw-gold-ere 0.7214 0.5630 0.6325 0.0676 0.3250 0.1119 0.3915 0.5292 0.4501 spa-nw-pred-ere 0.0000 0.0000 0.0000 0.0288 0.0376 0.0327 0.0238 0.0055 0.0090 ************************************************************* Run ID: Columbia_GW_sys2 Did the run access the live Web during the evaluation window: No Scores under each evaluation condition: language (cmn, eng, spa), genre (df, nw), input ere (gold-ere, predicted-ere). Scores were computed using lenient setting (full credit if and only if at least one correct provenance is returned). Condition Belief Sentiment All Prec Recall F1 Prec Recall F1 Prec Recall F1 cmn-df-gold-ere 0.7873 0.2569 0.3874 1.0000 0.0000 0.0000 0.7873 0.2194 0.3432 cmn-df-pred-ere 0.0020 0.0002 0.0003 1.0000 0.0000 0.0000 0.0020 0.0002 0.0003 cmn-nw-gold-ere 0.7847 0.1905 0.3065 1.0000 0.0000 0.0000 0.7847 0.1813 0.2946 cmn-nw-pred-ere 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 eng-df-gold-ere 0.8411 0.2790 0.4190 0.1152 0.4701 0.1851 0.2580 0.3266 0.2883 eng-df-pred-ere 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 eng-nw-gold-ere 0.8084 0.1551 0.2602 0.0798 0.1011 0.0892 0.4468 0.1481 0.2224 eng-nw-pred-ere 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 spa-df-gold-ere 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 spa-df-pred-ere 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 spa-nw-gold-ere 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 spa-nw-pred-ere 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 ************************************************************* Run ID: Columbia_GW_sys3 Did the run access the live Web during the evaluation window: No Scores under each evaluation condition: language (cmn, eng, spa), genre (df, nw), input ere (gold-ere, predicted-ere). Scores were computed using lenient setting (full credit if and only if at least one correct provenance is returned). Condition Belief Sentiment All Prec Recall F1 Prec Recall F1 Prec Recall F1 cmn-df-gold-ere 0.8067 0.8215 0.8140 1.0000 0.0000 0.0000 0.8067 0.7016 0.7505 cmn-df-pred-ere 0.0089 0.0022 0.0036 1.0000 0.0000 0.0000 0.0089 0.0019 0.0031 cmn-nw-gold-ere 0.8150 0.5198 0.6348 1.0000 0.0000 0.0000 0.8150 0.4949 0.6158 cmn-nw-pred-ere 0.0099 0.0018 0.0030 1.0000 0.0000 0.0000 0.0099 0.0017 0.0029 eng-df-gold-ere 0.7207 0.8419 0.7766 0.1603 0.3971 0.2284 0.4892 0.7310 0.5861 eng-df-pred-ere 0.0276 0.0035 0.0062 0.0974 0.0652 0.0781 0.0721 0.0190 0.0301 eng-nw-gold-ere 0.7741 0.5379 0.6348 0.0661 0.2575 0.1053 0.4518 0.5015 0.4753 eng-nw-pred-ere 0.0281 0.0045 0.0077 0.0217 0.0312 0.0256 0.0243 0.0082 0.0122 spa-df-gold-ere 0.7868 0.7568 0.7715 1.0000 0.0000 0.0000 0.7868 0.5613 0.6552 spa-df-pred-ere 0.0390 0.0024 0.0045 1.0000 0.0000 0.0000 0.0390 0.0018 0.0034 spa-nw-gold-ere 0.7995 0.5947 0.6820 1.0000 0.0000 0.0000 0.7995 0.5103 0.6229 spa-nw-pred-ere 0.0375 0.0035 0.0064 1.0000 0.0000 0.0000 0.0375 0.0030 0.0056