================================================================ TAC 2020 EPIDEMIC QUESTION ANSWERING EVALUATION RESULTS (TASK B) ================================================================ Team ID: HLTRI Organization(s): The University of Texas at Dallas; Human Language Technology Research Institute Run: HLTRI_1 Query Metric Score Median Maximum CQ101 NDNS-Exact 0.1503 0.1503 0.2700 CQ101 NDNS-Partial 0.1340 0.1340 0.2869 CQ101 NDNS-Relaxed 0.1456 0.1456 0.2894 CQ102 NDNS-Exact 0.2351 0.4063 0.7355 CQ102 NDNS-Partial 0.1922 0.3457 0.6011 CQ102 NDNS-Relaxed 0.1980 0.3494 0.6194 CQ103 NDNS-Exact 0.1247 0.1031 0.1767 CQ103 NDNS-Partial 0.1058 0.0875 0.1499 CQ103 NDNS-Relaxed 0.1058 0.0875 0.1499 CQ104 NDNS-Exact 0.0763 0.1333 0.4982 CQ104 NDNS-Partial 0.0763 0.1333 0.4982 CQ104 NDNS-Relaxed 0.0763 0.1333 0.4982 CQ105 NDNS-Exact 0.4100 0.3401 0.4124 CQ105 NDNS-Partial 0.3828 0.3572 0.4059 CQ105 NDNS-Relaxed 0.3828 0.3254 0.3850 CQ106 NDNS-Exact 0.4575 0.1063 0.5912 CQ106 NDNS-Partial 0.3731 0.0962 0.4821 CQ106 NDNS-Relaxed 0.3731 0.0962 0.4821 CQ107 NDNS-Exact 0.3541 0.2447 0.5969 CQ107 NDNS-Partial 0.2881 0.1991 0.4857 CQ107 NDNS-Relaxed 0.2881 0.1991 0.4857 CQ108 NDNS-Exact 0.2304 0.4565 0.7288 CQ108 NDNS-Partial 0.1923 0.4417 0.6084 CQ108 NDNS-Relaxed 0.1923 0.4100 0.6084 CQ109 NDNS-Exact 0.6309 0.3804 0.7500 CQ109 NDNS-Partial 0.6309 0.3867 0.7500 CQ109 NDNS-Relaxed 0.6309 0.3867 0.7500 CQ110 NDNS-Exact 0.5953 0.5001 0.5953 CQ110 NDNS-Partial 0.4240 0.3627 0.4440 CQ110 NDNS-Relaxed 0.4306 0.3654 0.4347 CQ111 NDNS-Exact 0.7144 0.5054 0.7144 CQ111 NDNS-Partial 0.6510 0.4649 0.6510 CQ111 NDNS-Relaxed 0.6510 0.4649 0.6510 CQ112 NDNS-Exact 0.2802 0.2450 0.3337 CQ112 NDNS-Partial 0.2435 0.2175 0.3522 CQ112 NDNS-Relaxed 0.2435 0.2175 0.3522 CQ113 NDNS-Exact 0.0603 0.0793 0.2236 CQ113 NDNS-Partial 0.0507 0.0667 0.1879 CQ113 NDNS-Relaxed 0.0507 0.0667 0.1879 CQ114 NDNS-Exact 0.5916 0.3065 0.7244 CQ114 NDNS-Partial 0.5602 0.2952 0.6859 CQ114 NDNS-Relaxed 0.5602 0.2902 0.6859 CQ115 NDNS-Exact 0.3979 0.4474 0.7055 CQ115 NDNS-Partial 0.3979 0.4699 0.8956 CQ115 NDNS-Relaxed 0.3979 0.4699 0.8956 CQ116 NDNS-Exact 0.1303 0.1467 0.1711 CQ116 NDNS-Partial 0.1074 0.1210 0.1411 CQ116 NDNS-Relaxed 0.1074 0.1210 0.1411 CQ117 NDNS-Exact 0.5007 0.2244 0.5007 CQ117 NDNS-Partial 0.5007 0.2244 0.5007 CQ117 NDNS-Relaxed 0.5007 0.2244 0.5007 CQ118 NDNS-Exact 0.6200 0.4771 0.6880 CQ118 NDNS-Partial 0.3949 0.3276 0.4794 CQ118 NDNS-Relaxed 0.3945 0.3087 0.4540 CQ119 NDNS-Exact 0.1554 0.1554 0.3275 CQ119 NDNS-Partial 0.1559 0.1559 0.3287 CQ119 NDNS-Relaxed 0.1559 0.1559 0.3287 CQ120 NDNS-Exact 0.4660 0.5451 0.8264 CQ120 NDNS-Partial 0.4322 0.5149 0.7665 CQ120 NDNS-Relaxed 0.4322 0.5149 0.7665 CQ121 NDNS-Exact 0.1641 0.2090 0.4916 CQ121 NDNS-Partial 0.1255 0.1776 0.3967 CQ121 NDNS-Relaxed 0.1286 0.1819 0.3927 CQ122 NDNS-Exact 0.6381 0.4885 0.8152 CQ122 NDNS-Partial 0.6381 0.5132 0.8152 CQ122 NDNS-Relaxed 0.6381 0.5132 0.8152 CQ123 NDNS-Exact 0.4532 0.5139 0.7226 CQ123 NDNS-Partial 0.3406 0.4122 0.5432 CQ123 NDNS-Relaxed 0.3429 0.4118 0.5468 CQ124 NDNS-Exact 0.5469 0.3348 0.6871 CQ124 NDNS-Partial 0.4430 0.2711 0.5566 CQ124 NDNS-Relaxed 0.4430 0.2711 0.5566 CQ125 NDNS-Exact 0.0000 0.0000 0.0449 CQ125 NDNS-Partial 0.0000 0.0000 0.0449 CQ125 NDNS-Relaxed 0.0000 0.0000 0.0449 CQ126 NDNS-Exact 0.2972 0.2838 0.2972 CQ126 NDNS-Partial 0.2505 0.2407 0.2743 CQ126 NDNS-Relaxed 0.2505 0.2407 0.2725 CQ127 NDNS-Exact 0.5740 0.1922 0.6145 CQ127 NDNS-Partial 0.4977 0.1703 0.5328 CQ127 NDNS-Relaxed 0.4977 0.1703 0.5328 CQ128 NDNS-Exact 0.1589 0.2830 0.5150 CQ128 NDNS-Partial 0.1448 0.2661 0.4694 CQ128 NDNS-Relaxed 0.1448 0.2655 0.4694 CQ129 NDNS-Exact 0.0205 0.0387 0.3125 CQ129 NDNS-Partial 0.0169 0.0318 0.2572 CQ129 NDNS-Relaxed 0.0169 0.0318 0.2572 CQ130 NDNS-Exact 0.3573 0.3557 0.6022 CQ130 NDNS-Partial 0.3573 0.3675 0.6022 CQ130 NDNS-Relaxed 0.3573 0.3675 0.6022 MEAN NDNS-Exact 0.3464 0.3142 0.4143 MEAN NDNS-Partial 0.3036 0.2858 0.3662 MEAN NDNS-Relaxed 0.3046 0.2845 0.3675 ******************************************************* Run: HLTRI_2 Query Metric Score Median Maximum CQ101 NDNS-Exact 0.2261 0.1503 0.2700 CQ101 NDNS-Partial 0.2015 0.1340 0.2869 CQ101 NDNS-Relaxed 0.2189 0.1456 0.2894 CQ102 NDNS-Exact 0.2234 0.4063 0.7355 CQ102 NDNS-Partial 0.1826 0.3457 0.6011 CQ102 NDNS-Relaxed 0.1881 0.3494 0.6194 CQ103 NDNS-Exact 0.0961 0.1031 0.1767 CQ103 NDNS-Partial 0.0816 0.0875 0.1499 CQ103 NDNS-Relaxed 0.0816 0.0875 0.1499 CQ104 NDNS-Exact 0.1357 0.1333 0.4982 CQ104 NDNS-Partial 0.1357 0.1333 0.4982 CQ104 NDNS-Relaxed 0.1357 0.1333 0.4982 CQ105 NDNS-Exact 0.4082 0.3401 0.4124 CQ105 NDNS-Partial 0.3811 0.3572 0.4059 CQ105 NDNS-Relaxed 0.3811 0.3254 0.3850 CQ106 NDNS-Exact 0.4556 0.1063 0.5912 CQ106 NDNS-Partial 0.3715 0.0962 0.4821 CQ106 NDNS-Relaxed 0.3715 0.0962 0.4821 CQ107 NDNS-Exact 0.3529 0.2447 0.5969 CQ107 NDNS-Partial 0.2872 0.1991 0.4857 CQ107 NDNS-Relaxed 0.2872 0.1991 0.4857 CQ108 NDNS-Exact 0.2075 0.4565 0.7288 CQ108 NDNS-Partial 0.1732 0.4417 0.6084 CQ108 NDNS-Relaxed 0.1732 0.4100 0.6084 CQ109 NDNS-Exact 0.6934 0.3804 0.7500 CQ109 NDNS-Partial 0.6934 0.3867 0.7500 CQ109 NDNS-Relaxed 0.6934 0.3867 0.7500 CQ110 NDNS-Exact 0.4419 0.5001 0.5953 CQ110 NDNS-Partial 0.3147 0.3627 0.4440 CQ110 NDNS-Relaxed 0.3196 0.3654 0.4347 CQ111 NDNS-Exact 0.7116 0.5054 0.7144 CQ111 NDNS-Partial 0.6485 0.4649 0.6510 CQ111 NDNS-Relaxed 0.6485 0.4649 0.6510 CQ112 NDNS-Exact 0.2595 0.2450 0.3337 CQ112 NDNS-Partial 0.2255 0.2175 0.3522 CQ112 NDNS-Relaxed 0.2255 0.2175 0.3522 CQ113 NDNS-Exact 0.0633 0.0793 0.2236 CQ113 NDNS-Partial 0.0532 0.0667 0.1879 CQ113 NDNS-Relaxed 0.0532 0.0667 0.1879 CQ114 NDNS-Exact 0.7244 0.3065 0.7244 CQ114 NDNS-Partial 0.6859 0.2952 0.6859 CQ114 NDNS-Relaxed 0.6859 0.2902 0.6859 CQ115 NDNS-Exact 0.3882 0.4474 0.7055 CQ115 NDNS-Partial 0.3882 0.4699 0.8956 CQ115 NDNS-Relaxed 0.3882 0.4699 0.8956 CQ116 NDNS-Exact 0.1401 0.1467 0.1711 CQ116 NDNS-Partial 0.1155 0.1210 0.1411 CQ116 NDNS-Relaxed 0.1155 0.1210 0.1411 CQ117 NDNS-Exact 0.4918 0.2244 0.5007 CQ117 NDNS-Partial 0.4918 0.2244 0.5007 CQ117 NDNS-Relaxed 0.4918 0.2244 0.5007 CQ118 NDNS-Exact 0.6151 0.4771 0.6880 CQ118 NDNS-Partial 0.3917 0.3276 0.4794 CQ118 NDNS-Relaxed 0.3914 0.3087 0.4540 CQ119 NDNS-Exact 0.1554 0.1554 0.3275 CQ119 NDNS-Partial 0.1559 0.1559 0.3287 CQ119 NDNS-Relaxed 0.1559 0.1559 0.3287 CQ120 NDNS-Exact 0.7313 0.5451 0.8264 CQ120 NDNS-Partial 0.6782 0.5149 0.7665 CQ120 NDNS-Relaxed 0.6782 0.5149 0.7665 CQ121 NDNS-Exact 0.1559 0.2090 0.4916 CQ121 NDNS-Partial 0.1193 0.1776 0.3967 CQ121 NDNS-Relaxed 0.1222 0.1819 0.3927 CQ122 NDNS-Exact 0.6530 0.4885 0.8152 CQ122 NDNS-Partial 0.6530 0.5132 0.8152 CQ122 NDNS-Relaxed 0.6530 0.5132 0.8152 CQ123 NDNS-Exact 0.4359 0.5139 0.7226 CQ123 NDNS-Partial 0.3276 0.4122 0.5432 CQ123 NDNS-Relaxed 0.3298 0.4118 0.5468 CQ124 NDNS-Exact 0.5469 0.3348 0.6871 CQ124 NDNS-Partial 0.4430 0.2711 0.5566 CQ124 NDNS-Relaxed 0.4430 0.2711 0.5566 CQ125 NDNS-Exact 0.0000 0.0000 0.0449 CQ125 NDNS-Partial 0.0000 0.0000 0.0449 CQ125 NDNS-Relaxed 0.0000 0.0000 0.0449 CQ126 NDNS-Exact 0.2958 0.2838 0.2972 CQ126 NDNS-Partial 0.2493 0.2407 0.2743 CQ126 NDNS-Relaxed 0.2493 0.2407 0.2725 CQ127 NDNS-Exact 0.4026 0.1922 0.6145 CQ127 NDNS-Partial 0.3490 0.1703 0.5328 CQ127 NDNS-Relaxed 0.3490 0.1703 0.5328 CQ128 NDNS-Exact 0.1801 0.2830 0.5150 CQ128 NDNS-Partial 0.1642 0.2661 0.4694 CQ128 NDNS-Relaxed 0.1642 0.2655 0.4694 CQ129 NDNS-Exact 0.0206 0.0387 0.3125 CQ129 NDNS-Partial 0.0169 0.0318 0.2572 CQ129 NDNS-Relaxed 0.0169 0.0318 0.2572 CQ130 NDNS-Exact 0.3776 0.3557 0.6022 CQ130 NDNS-Partial 0.3776 0.3675 0.6022 CQ130 NDNS-Relaxed 0.3776 0.3675 0.6022 MEAN NDNS-Exact 0.3530 0.3142 0.4143 MEAN NDNS-Partial 0.3119 0.2858 0.3662 MEAN NDNS-Relaxed 0.3130 0.2845 0.3675 ******************************************************* Run: HLTRI_3 Query Metric Score Median Maximum CQ101 NDNS-Exact 0.1503 0.1503 0.2700 CQ101 NDNS-Partial 0.1340 0.1340 0.2869 CQ101 NDNS-Relaxed 0.1456 0.1456 0.2894 CQ102 NDNS-Exact 0.3261 0.4063 0.7355 CQ102 NDNS-Partial 0.2665 0.3457 0.6011 CQ102 NDNS-Relaxed 0.2746 0.3494 0.6194 CQ103 NDNS-Exact 0.1767 0.1031 0.1767 CQ103 NDNS-Partial 0.1499 0.0875 0.1499 CQ103 NDNS-Relaxed 0.1499 0.0875 0.1499 CQ104 NDNS-Exact 0.1233 0.1333 0.4982 CQ104 NDNS-Partial 0.1233 0.1333 0.4982 CQ104 NDNS-Relaxed 0.1233 0.1333 0.4982 CQ105 NDNS-Exact 0.4124 0.3401 0.4124 CQ105 NDNS-Partial 0.3850 0.3572 0.4059 CQ105 NDNS-Relaxed 0.3850 0.3254 0.3850 CQ106 NDNS-Exact 0.5912 0.1063 0.5912 CQ106 NDNS-Partial 0.4821 0.0962 0.4821 CQ106 NDNS-Relaxed 0.4821 0.0962 0.4821 CQ107 NDNS-Exact 0.5969 0.2447 0.5969 CQ107 NDNS-Partial 0.4857 0.1991 0.4857 CQ107 NDNS-Relaxed 0.4857 0.1991 0.4857 CQ108 NDNS-Exact 0.4949 0.4565 0.7288 CQ108 NDNS-Partial 0.4131 0.4417 0.6084 CQ108 NDNS-Relaxed 0.4131 0.4100 0.6084 CQ109 NDNS-Exact 0.5089 0.3804 0.7500 CQ109 NDNS-Partial 0.5089 0.3867 0.7500 CQ109 NDNS-Relaxed 0.5089 0.3867 0.7500 CQ110 NDNS-Exact 0.5631 0.5001 0.5953 CQ110 NDNS-Partial 0.4011 0.3627 0.4440 CQ110 NDNS-Relaxed 0.4072 0.3654 0.4347 CQ111 NDNS-Exact 0.7116 0.5054 0.7144 CQ111 NDNS-Partial 0.6485 0.4649 0.6510 CQ111 NDNS-Relaxed 0.6485 0.4649 0.6510 CQ112 NDNS-Exact 0.3337 0.2450 0.3337 CQ112 NDNS-Partial 0.2900 0.2175 0.3522 CQ112 NDNS-Relaxed 0.2900 0.2175 0.3522 CQ113 NDNS-Exact 0.0627 0.0793 0.2236 CQ113 NDNS-Partial 0.0527 0.0667 0.1879 CQ113 NDNS-Relaxed 0.0527 0.0667 0.1879 CQ114 NDNS-Exact 0.3304 0.3065 0.7244 CQ114 NDNS-Partial 0.3128 0.2952 0.6859 CQ114 NDNS-Relaxed 0.3128 0.2902 0.6859 CQ115 NDNS-Exact 0.4448 0.4474 0.7055 CQ115 NDNS-Partial 0.4448 0.4699 0.8956 CQ115 NDNS-Relaxed 0.4448 0.4699 0.8956 CQ116 NDNS-Exact 0.1375 0.1467 0.1711 CQ116 NDNS-Partial 0.1134 0.1210 0.1411 CQ116 NDNS-Relaxed 0.1134 0.1210 0.1411 CQ117 NDNS-Exact 0.4618 0.2244 0.5007 CQ117 NDNS-Partial 0.4618 0.2244 0.5007 CQ117 NDNS-Relaxed 0.4618 0.2244 0.5007 CQ118 NDNS-Exact 0.6880 0.4771 0.6880 CQ118 NDNS-Partial 0.4381 0.3276 0.4794 CQ118 NDNS-Relaxed 0.4377 0.3087 0.4540 CQ119 NDNS-Exact 0.1554 0.1554 0.3275 CQ119 NDNS-Partial 0.1559 0.1559 0.3287 CQ119 NDNS-Relaxed 0.1559 0.1559 0.3287 CQ120 NDNS-Exact 0.4731 0.5451 0.8264 CQ120 NDNS-Partial 0.4388 0.5149 0.7665 CQ120 NDNS-Relaxed 0.4388 0.5149 0.7665 CQ121 NDNS-Exact 0.1679 0.2090 0.4916 CQ121 NDNS-Partial 0.1285 0.1776 0.3967 CQ121 NDNS-Relaxed 0.1316 0.1819 0.3927 CQ122 NDNS-Exact 0.5307 0.4885 0.8152 CQ122 NDNS-Partial 0.5307 0.5132 0.8152 CQ122 NDNS-Relaxed 0.5307 0.5132 0.8152 CQ123 NDNS-Exact 0.5416 0.5139 0.7226 CQ123 NDNS-Partial 0.4071 0.4122 0.5432 CQ123 NDNS-Relaxed 0.4098 0.4118 0.5468 CQ124 NDNS-Exact 0.3734 0.3348 0.6871 CQ124 NDNS-Partial 0.3025 0.2711 0.5566 CQ124 NDNS-Relaxed 0.3025 0.2711 0.5566 CQ125 NDNS-Exact 0.0000 0.0000 0.0449 CQ125 NDNS-Partial 0.0000 0.0000 0.0449 CQ125 NDNS-Relaxed 0.0000 0.0000 0.0449 CQ126 NDNS-Exact 0.2841 0.2838 0.2972 CQ126 NDNS-Partial 0.2395 0.2407 0.2743 CQ126 NDNS-Relaxed 0.2395 0.2407 0.2725 CQ127 NDNS-Exact 0.6145 0.1922 0.6145 CQ127 NDNS-Partial 0.5328 0.1703 0.5328 CQ127 NDNS-Relaxed 0.5328 0.1703 0.5328 CQ128 NDNS-Exact 0.1787 0.2830 0.5150 CQ128 NDNS-Partial 0.1629 0.2661 0.4694 CQ128 NDNS-Relaxed 0.1629 0.2655 0.4694 CQ129 NDNS-Exact 0.0206 0.0387 0.3125 CQ129 NDNS-Partial 0.0169 0.0318 0.2572 CQ129 NDNS-Relaxed 0.0169 0.0318 0.2572 CQ130 NDNS-Exact 0.4458 0.3557 0.6022 CQ130 NDNS-Partial 0.4458 0.3675 0.6022 CQ130 NDNS-Relaxed 0.4458 0.3675 0.6022 MEAN NDNS-Exact 0.3633 0.3142 0.4143 MEAN NDNS-Partial 0.3158 0.2858 0.3662 MEAN NDNS-Relaxed 0.3168 0.2845 0.3675 *******************************************************