================================================================ TAC 2020 EPIDEMIC QUESTION ANSWERING EVALUATION RESULTS (TASK A) ================================================================ Team ID: h2oloo Organization(s): University of Waterloo Run: h2oloo_1 Query Metric Score Median Maximum EQ101 NDNS-Exact 0.5083 0.4974 0.6811 EQ101 NDNS-Partial 0.4207 0.4186 0.5638 EQ101 NDNS-Relaxed 0.4207 0.4186 0.5638 EQ102 NDNS-Exact 0.4754 0.3410 0.5418 EQ102 NDNS-Partial 0.4343 0.3115 0.4950 EQ102 NDNS-Relaxed 0.4343 0.3115 0.4950 EQ103 NDNS-Exact 0.3636 0.2500 0.3667 EQ103 NDNS-Partial 0.3023 0.2078 0.3048 EQ103 NDNS-Relaxed 0.3023 0.2078 0.3048 EQ104 NDNS-Exact 0.4508 0.4219 0.6273 EQ104 NDNS-Partial 0.4202 0.4080 0.6397 EQ104 NDNS-Relaxed 0.4202 0.4080 0.5977 EQ105 NDNS-Exact 0.1894 0.2766 0.7001 EQ105 NDNS-Partial 0.1790 0.2784 0.6617 EQ105 NDNS-Relaxed 0.1790 0.2784 0.6617 EQ106 NDNS-Exact 0.4080 0.4169 0.6451 EQ106 NDNS-Partial 0.3305 0.3377 0.5332 EQ106 NDNS-Relaxed 0.3305 0.3377 0.5225 EQ107 NDNS-Exact 0.1195 0.1240 0.1839 EQ107 NDNS-Partial 0.1101 0.1144 0.1695 EQ107 NDNS-Relaxed 0.1101 0.1144 0.1695 EQ108 NDNS-Exact 0.3784 0.3949 0.6287 EQ108 NDNS-Partial 0.3239 0.3380 0.5382 EQ108 NDNS-Relaxed 0.3239 0.3380 0.5382 EQ109 NDNS-Exact 0.5655 0.6309 1.0000 EQ109 NDNS-Partial 0.5655 0.7500 1.0000 EQ109 NDNS-Relaxed 0.5655 0.7500 1.0000 EQ110 NDNS-Exact 0.5436 0.3887 0.5470 EQ110 NDNS-Partial 0.4227 0.3022 0.4270 EQ110 NDNS-Relaxed 0.4184 0.2992 0.4210 EQ111 NDNS-Exact 0.4118 0.4480 0.5874 EQ111 NDNS-Partial 0.3231 0.3620 0.5094 EQ111 NDNS-Relaxed 0.3321 0.3688 0.5236 EQ112 NDNS-Exact 0.3790 0.5012 0.5953 EQ112 NDNS-Partial 0.3281 0.4338 0.5153 EQ112 NDNS-Relaxed 0.3281 0.4338 0.5153 EQ113 NDNS-Exact 0.3296 0.3610 0.5062 EQ113 NDNS-Partial 0.3205 0.3511 0.4923 EQ113 NDNS-Relaxed 0.3205 0.3511 0.4923 EQ114 NDNS-Exact 0.3283 0.3145 0.6719 EQ114 NDNS-Partial 0.3172 0.3039 0.6492 EQ114 NDNS-Relaxed 0.3172 0.3039 0.6492 EQ115 NDNS-Exact 0.0779 0.1533 0.2688 EQ115 NDNS-Partial 0.0742 0.1461 0.2561 EQ115 NDNS-Relaxed 0.0744 0.1466 0.2570 EQ116 NDNS-Exact 0.1566 0.2327 0.3331 EQ116 NDNS-Partial 0.1386 0.2058 0.2946 EQ116 NDNS-Relaxed 0.1386 0.2058 0.2946 EQ117 NDNS-Exact 0.3395 0.3511 0.5322 EQ117 NDNS-Partial 0.3390 0.3715 0.6085 EQ117 NDNS-Relaxed 0.3390 0.3715 0.6085 EQ118 NDNS-Exact 0.6529 0.5646 0.6908 EQ118 NDNS-Partial 0.5042 0.4360 0.5335 EQ118 NDNS-Relaxed 0.5042 0.4360 0.5335 EQ119 NDNS-Exact 0.2976 0.2868 0.4027 EQ119 NDNS-Partial 0.2266 0.2263 0.3067 EQ119 NDNS-Relaxed 0.2266 0.2244 0.3067 EQ120 NDNS-Exact 0.7176 0.5027 0.7176 EQ120 NDNS-Partial 0.6767 0.4740 0.6767 EQ120 NDNS-Relaxed 0.6849 0.4798 0.6849 EQ121 NDNS-Exact 0.4689 0.3903 0.4689 EQ121 NDNS-Partial 0.4589 0.3819 0.4589 EQ121 NDNS-Relaxed 0.4589 0.3819 0.4589 EQ122 NDNS-Exact 0.1890 0.2465 0.3862 EQ122 NDNS-Partial 0.1578 0.2078 0.3409 EQ122 NDNS-Relaxed 0.1588 0.2092 0.3244 EQ123 NDNS-Exact 0.4964 0.5079 0.6317 EQ123 NDNS-Partial 0.3845 0.3934 0.4892 EQ123 NDNS-Relaxed 0.3854 0.3943 0.4904 EQ124 NDNS-Exact 0.6063 0.4694 0.6163 EQ124 NDNS-Partial 0.5212 0.4035 0.5298 EQ124 NDNS-Relaxed 0.5212 0.4035 0.5298 EQ125 NDNS-Exact 0.4126 0.3250 0.8570 EQ125 NDNS-Partial 0.3467 0.2731 0.7201 EQ125 NDNS-Relaxed 0.3467 0.2731 0.7201 EQ126 NDNS-Exact 0.3204 0.2966 0.6099 EQ126 NDNS-Partial 0.2588 0.2396 0.4927 EQ126 NDNS-Relaxed 0.2602 0.2408 0.4952 EQ127 NDNS-Exact 0.4402 0.4034 0.5870 EQ127 NDNS-Partial 0.3794 0.3684 0.5060 EQ127 NDNS-Relaxed 0.3870 0.3737 0.5160 EQ128 NDNS-Exact 0.3416 0.3416 0.4318 EQ128 NDNS-Partial 0.2533 0.2551 0.3393 EQ128 NDNS-Relaxed 0.2555 0.2555 0.3331 EQ129 NDNS-Exact 0.2675 0.1448 0.3360 EQ129 NDNS-Partial 0.2227 0.1205 0.2868 EQ129 NDNS-Relaxed 0.2227 0.1205 0.2868 EQ130 NDNS-Exact 0.4035 0.4404 0.6946 EQ130 NDNS-Partial 0.4035 0.4535 0.6946 EQ130 NDNS-Relaxed 0.4035 0.4535 0.6946 MEAN NDNS-Exact 0.3880 0.3802 0.4207 MEAN NDNS-Partial 0.3381 0.3377 0.3699 MEAN NDNS-Relaxed 0.3390 0.3387 0.3709 ******************************************************* Run: h2oloo_2 Query Metric Score Median Maximum EQ101 NDNS-Exact 0.5031 0.4974 0.6811 EQ101 NDNS-Partial 0.4164 0.4186 0.5638 EQ101 NDNS-Relaxed 0.4164 0.4186 0.5638 EQ102 NDNS-Exact 0.4941 0.3410 0.5418 EQ102 NDNS-Partial 0.4514 0.3115 0.4950 EQ102 NDNS-Relaxed 0.4514 0.3115 0.4950 EQ103 NDNS-Exact 0.3667 0.2500 0.3667 EQ103 NDNS-Partial 0.3048 0.2078 0.3048 EQ103 NDNS-Relaxed 0.3048 0.2078 0.3048 EQ104 NDNS-Exact 0.4504 0.4219 0.6273 EQ104 NDNS-Partial 0.4199 0.4080 0.6397 EQ104 NDNS-Relaxed 0.4199 0.4080 0.5977 EQ105 NDNS-Exact 0.1925 0.2766 0.7001 EQ105 NDNS-Partial 0.1820 0.2784 0.6617 EQ105 NDNS-Relaxed 0.1820 0.2784 0.6617 EQ106 NDNS-Exact 0.4112 0.4169 0.6451 EQ106 NDNS-Partial 0.3331 0.3377 0.5332 EQ106 NDNS-Relaxed 0.3331 0.3377 0.5225 EQ107 NDNS-Exact 0.1200 0.1240 0.1839 EQ107 NDNS-Partial 0.1106 0.1144 0.1695 EQ107 NDNS-Relaxed 0.1106 0.1144 0.1695 EQ108 NDNS-Exact 0.3784 0.3949 0.6287 EQ108 NDNS-Partial 0.3239 0.3380 0.5382 EQ108 NDNS-Relaxed 0.3239 0.3380 0.5382 EQ109 NDNS-Exact 0.6309 0.6309 1.0000 EQ109 NDNS-Partial 0.6309 0.7500 1.0000 EQ109 NDNS-Relaxed 0.6309 0.7500 1.0000 EQ110 NDNS-Exact 0.5470 0.3887 0.5470 EQ110 NDNS-Partial 0.4253 0.3022 0.4270 EQ110 NDNS-Relaxed 0.4210 0.2992 0.4210 EQ111 NDNS-Exact 0.4106 0.4480 0.5874 EQ111 NDNS-Partial 0.3221 0.3620 0.5094 EQ111 NDNS-Relaxed 0.3310 0.3688 0.5236 EQ112 NDNS-Exact 0.3797 0.5012 0.5953 EQ112 NDNS-Partial 0.3286 0.4338 0.5153 EQ112 NDNS-Relaxed 0.3286 0.4338 0.5153 EQ113 NDNS-Exact 0.3394 0.3610 0.5062 EQ113 NDNS-Partial 0.3301 0.3511 0.4923 EQ113 NDNS-Relaxed 0.3301 0.3511 0.4923 EQ114 NDNS-Exact 0.3283 0.3145 0.6719 EQ114 NDNS-Partial 0.3172 0.3039 0.6492 EQ114 NDNS-Relaxed 0.3172 0.3039 0.6492 EQ115 NDNS-Exact 0.0779 0.1533 0.2688 EQ115 NDNS-Partial 0.0742 0.1461 0.2561 EQ115 NDNS-Relaxed 0.0744 0.1466 0.2570 EQ116 NDNS-Exact 0.1574 0.2327 0.3331 EQ116 NDNS-Partial 0.1392 0.2058 0.2946 EQ116 NDNS-Relaxed 0.1392 0.2058 0.2946 EQ117 NDNS-Exact 0.3395 0.3511 0.5322 EQ117 NDNS-Partial 0.3389 0.3715 0.6085 EQ117 NDNS-Relaxed 0.3389 0.3715 0.6085 EQ118 NDNS-Exact 0.6523 0.5646 0.6908 EQ118 NDNS-Partial 0.5037 0.4360 0.5335 EQ118 NDNS-Relaxed 0.5037 0.4360 0.5335 EQ119 NDNS-Exact 0.2989 0.2868 0.4027 EQ119 NDNS-Partial 0.2276 0.2263 0.3067 EQ119 NDNS-Relaxed 0.2276 0.2244 0.3067 EQ120 NDNS-Exact 0.7174 0.5027 0.7176 EQ120 NDNS-Partial 0.6764 0.4740 0.6767 EQ120 NDNS-Relaxed 0.6846 0.4798 0.6849 EQ121 NDNS-Exact 0.4689 0.3903 0.4689 EQ121 NDNS-Partial 0.4589 0.3819 0.4589 EQ121 NDNS-Relaxed 0.4589 0.3819 0.4589 EQ122 NDNS-Exact 0.1874 0.2465 0.3862 EQ122 NDNS-Partial 0.1564 0.2078 0.3409 EQ122 NDNS-Relaxed 0.1574 0.2092 0.3244 EQ123 NDNS-Exact 0.4961 0.5079 0.6317 EQ123 NDNS-Partial 0.3842 0.3934 0.4892 EQ123 NDNS-Relaxed 0.3851 0.3943 0.4904 EQ124 NDNS-Exact 0.6163 0.4694 0.6163 EQ124 NDNS-Partial 0.5298 0.4035 0.5298 EQ124 NDNS-Relaxed 0.5298 0.4035 0.5298 EQ125 NDNS-Exact 0.4114 0.3250 0.8570 EQ125 NDNS-Partial 0.3457 0.2731 0.7201 EQ125 NDNS-Relaxed 0.3457 0.2731 0.7201 EQ126 NDNS-Exact 0.2728 0.2966 0.6099 EQ126 NDNS-Partial 0.2203 0.2396 0.4927 EQ126 NDNS-Relaxed 0.2215 0.2408 0.4952 EQ127 NDNS-Exact 0.4404 0.4034 0.5870 EQ127 NDNS-Partial 0.3796 0.3684 0.5060 EQ127 NDNS-Relaxed 0.3871 0.3737 0.5160 EQ128 NDNS-Exact 0.3416 0.3416 0.4318 EQ128 NDNS-Partial 0.2533 0.2551 0.3393 EQ128 NDNS-Relaxed 0.2555 0.2555 0.3331 EQ129 NDNS-Exact 0.2675 0.1448 0.3360 EQ129 NDNS-Partial 0.2227 0.1205 0.2868 EQ129 NDNS-Relaxed 0.2227 0.1205 0.2868 EQ130 NDNS-Exact 0.4037 0.4404 0.6946 EQ130 NDNS-Partial 0.4037 0.4535 0.6946 EQ130 NDNS-Relaxed 0.4037 0.4535 0.6946 MEAN NDNS-Exact 0.3901 0.3802 0.4207 MEAN NDNS-Partial 0.3404 0.3377 0.3699 MEAN NDNS-Relaxed 0.3412 0.3387 0.3709 ******************************************************* Run: h2oloo_3 Query Metric Score Median Maximum EQ101 NDNS-Exact 0.4917 0.4974 0.6811 EQ101 NDNS-Partial 0.4070 0.4186 0.5638 EQ101 NDNS-Relaxed 0.4070 0.4186 0.5638 EQ102 NDNS-Exact 0.4941 0.3410 0.5418 EQ102 NDNS-Partial 0.4514 0.3115 0.4950 EQ102 NDNS-Relaxed 0.4514 0.3115 0.4950 EQ103 NDNS-Exact 0.3628 0.2500 0.3667 EQ103 NDNS-Partial 0.3016 0.2078 0.3048 EQ103 NDNS-Relaxed 0.3016 0.2078 0.3048 EQ104 NDNS-Exact 0.3934 0.4219 0.6273 EQ104 NDNS-Partial 0.3667 0.4080 0.6397 EQ104 NDNS-Relaxed 0.3667 0.4080 0.5977 EQ105 NDNS-Exact 0.1913 0.2766 0.7001 EQ105 NDNS-Partial 0.1808 0.2784 0.6617 EQ105 NDNS-Relaxed 0.1808 0.2784 0.6617 EQ106 NDNS-Exact 0.4016 0.4169 0.6451 EQ106 NDNS-Partial 0.3253 0.3377 0.5332 EQ106 NDNS-Relaxed 0.3253 0.3377 0.5225 EQ107 NDNS-Exact 0.1111 0.1240 0.1839 EQ107 NDNS-Partial 0.1024 0.1144 0.1695 EQ107 NDNS-Relaxed 0.1024 0.1144 0.1695 EQ108 NDNS-Exact 0.3282 0.3949 0.6287 EQ108 NDNS-Partial 0.2809 0.3380 0.5382 EQ108 NDNS-Relaxed 0.2809 0.3380 0.5382 EQ109 NDNS-Exact 0.6309 0.6309 1.0000 EQ109 NDNS-Partial 0.6309 0.7500 1.0000 EQ109 NDNS-Relaxed 0.6309 0.7500 1.0000 EQ110 NDNS-Exact 0.5282 0.3887 0.5470 EQ110 NDNS-Partial 0.4106 0.3022 0.4270 EQ110 NDNS-Relaxed 0.4065 0.2992 0.4210 EQ111 NDNS-Exact 0.3129 0.4480 0.5874 EQ111 NDNS-Partial 0.2455 0.3620 0.5094 EQ111 NDNS-Relaxed 0.2523 0.3688 0.5236 EQ112 NDNS-Exact 0.3829 0.5012 0.5953 EQ112 NDNS-Partial 0.3314 0.4338 0.5153 EQ112 NDNS-Relaxed 0.3314 0.4338 0.5153 EQ113 NDNS-Exact 0.3287 0.3610 0.5062 EQ113 NDNS-Partial 0.3197 0.3511 0.4923 EQ113 NDNS-Relaxed 0.3197 0.3511 0.4923 EQ114 NDNS-Exact 0.3076 0.3145 0.6719 EQ114 NDNS-Partial 0.2973 0.3039 0.6492 EQ114 NDNS-Relaxed 0.2973 0.3039 0.6492 EQ115 NDNS-Exact 0.0947 0.1533 0.2688 EQ115 NDNS-Partial 0.0903 0.1461 0.2561 EQ115 NDNS-Relaxed 0.0906 0.1466 0.2570 EQ116 NDNS-Exact 0.2082 0.2327 0.3331 EQ116 NDNS-Partial 0.1842 0.2058 0.2946 EQ116 NDNS-Relaxed 0.1842 0.2058 0.2946 EQ117 NDNS-Exact 0.3202 0.3511 0.5322 EQ117 NDNS-Partial 0.3196 0.3715 0.6085 EQ117 NDNS-Relaxed 0.3196 0.3715 0.6085 EQ118 NDNS-Exact 0.5784 0.5646 0.6908 EQ118 NDNS-Partial 0.4466 0.4360 0.5335 EQ118 NDNS-Relaxed 0.4466 0.4360 0.5335 EQ119 NDNS-Exact 0.2725 0.2868 0.4027 EQ119 NDNS-Partial 0.2075 0.2263 0.3067 EQ119 NDNS-Relaxed 0.2075 0.2244 0.3067 EQ120 NDNS-Exact 0.6924 0.5027 0.7176 EQ120 NDNS-Partial 0.6529 0.4740 0.6767 EQ120 NDNS-Relaxed 0.6608 0.4798 0.6849 EQ121 NDNS-Exact 0.4689 0.3903 0.4689 EQ121 NDNS-Partial 0.4589 0.3819 0.4589 EQ121 NDNS-Relaxed 0.4589 0.3819 0.4589 EQ122 NDNS-Exact 0.1734 0.2465 0.3862 EQ122 NDNS-Partial 0.1447 0.2078 0.3409 EQ122 NDNS-Relaxed 0.1456 0.2092 0.3244 EQ123 NDNS-Exact 0.4482 0.5079 0.6317 EQ123 NDNS-Partial 0.3471 0.3934 0.4892 EQ123 NDNS-Relaxed 0.3479 0.3943 0.4904 EQ124 NDNS-Exact 0.5460 0.4694 0.6163 EQ124 NDNS-Partial 0.4694 0.4035 0.5298 EQ124 NDNS-Relaxed 0.4694 0.4035 0.5298 EQ125 NDNS-Exact 0.4092 0.3250 0.8570 EQ125 NDNS-Partial 0.3438 0.2731 0.7201 EQ125 NDNS-Relaxed 0.3438 0.2731 0.7201 EQ126 NDNS-Exact 0.3285 0.2966 0.6099 EQ126 NDNS-Partial 0.2653 0.2396 0.4927 EQ126 NDNS-Relaxed 0.2667 0.2408 0.4952 EQ127 NDNS-Exact 0.4430 0.4034 0.5870 EQ127 NDNS-Partial 0.3818 0.3684 0.5060 EQ127 NDNS-Relaxed 0.3894 0.3737 0.5160 EQ128 NDNS-Exact 0.3464 0.3416 0.4318 EQ128 NDNS-Partial 0.2568 0.2551 0.3393 EQ128 NDNS-Relaxed 0.2591 0.2555 0.3331 EQ129 NDNS-Exact 0.2250 0.1448 0.3360 EQ129 NDNS-Partial 0.1873 0.1205 0.2868 EQ129 NDNS-Relaxed 0.1873 0.1205 0.2868 EQ130 NDNS-Exact 0.4448 0.4404 0.6946 EQ130 NDNS-Partial 0.4448 0.4535 0.6946 EQ130 NDNS-Relaxed 0.4448 0.4535 0.6946 MEAN NDNS-Exact 0.3755 0.3802 0.4207 MEAN NDNS-Partial 0.3284 0.3377 0.3699 MEAN NDNS-Relaxed 0.3292 0.3387 0.3709 *******************************************************