================================================================ TAC 2020 EPIDEMIC QUESTION ANSWERING EVALUATION RESULTS (TASK A) ================================================================ Team ID: vigicovid Organization(s): UNED; UPV/EHU; Elhuyar Run: vigicovid_1 Query Metric Score Median Maximum EQ101 NDNS-Exact 0.2795 0.4974 0.6811 EQ101 NDNS-Partial 0.2313 0.4186 0.5638 EQ101 NDNS-Relaxed 0.2313 0.4186 0.5638 EQ102 NDNS-Exact 0.2688 0.3410 0.5418 EQ102 NDNS-Partial 0.2456 0.3115 0.4950 EQ102 NDNS-Relaxed 0.2456 0.3115 0.4950 EQ103 NDNS-Exact 0.2426 0.2500 0.3667 EQ103 NDNS-Partial 0.2016 0.2078 0.3048 EQ103 NDNS-Relaxed 0.2016 0.2078 0.3048 EQ104 NDNS-Exact 0.3762 0.4219 0.6273 EQ104 NDNS-Partial 0.3507 0.4080 0.6397 EQ104 NDNS-Relaxed 0.3507 0.4080 0.5977 EQ105 NDNS-Exact 0.4648 0.2766 0.7001 EQ105 NDNS-Partial 0.4393 0.2784 0.6617 EQ105 NDNS-Relaxed 0.4393 0.2784 0.6617 EQ106 NDNS-Exact 0.2942 0.4169 0.6451 EQ106 NDNS-Partial 0.2383 0.3377 0.5332 EQ106 NDNS-Relaxed 0.2383 0.3377 0.5225 EQ107 NDNS-Exact 0.1339 0.1240 0.1839 EQ107 NDNS-Partial 0.1235 0.1144 0.1695 EQ107 NDNS-Relaxed 0.1235 0.1144 0.1695 EQ108 NDNS-Exact 0.5957 0.3949 0.6287 EQ108 NDNS-Partial 0.5099 0.3380 0.5382 EQ108 NDNS-Relaxed 0.5099 0.3380 0.5382 EQ109 NDNS-Exact 0.5000 0.6309 1.0000 EQ109 NDNS-Partial 0.5000 0.7500 1.0000 EQ109 NDNS-Relaxed 0.5000 0.7500 1.0000 EQ110 NDNS-Exact 0.5101 0.3887 0.5470 EQ110 NDNS-Partial 0.4017 0.3022 0.4270 EQ110 NDNS-Relaxed 0.3926 0.2992 0.4210 EQ111 NDNS-Exact 0.4824 0.4480 0.5874 EQ111 NDNS-Partial 0.3785 0.3620 0.5094 EQ111 NDNS-Relaxed 0.3890 0.3688 0.5236 EQ112 NDNS-Exact 0.5443 0.5012 0.5953 EQ112 NDNS-Partial 0.4712 0.4338 0.5153 EQ112 NDNS-Relaxed 0.4712 0.4338 0.5153 EQ113 NDNS-Exact 0.4715 0.3610 0.5062 EQ113 NDNS-Partial 0.4586 0.3511 0.4923 EQ113 NDNS-Relaxed 0.4586 0.3511 0.4923 EQ114 NDNS-Exact 0.3213 0.3145 0.6719 EQ114 NDNS-Partial 0.3105 0.3039 0.6492 EQ114 NDNS-Relaxed 0.3105 0.3039 0.6492 EQ115 NDNS-Exact 0.1498 0.1533 0.2688 EQ115 NDNS-Partial 0.1428 0.1461 0.2561 EQ115 NDNS-Relaxed 0.1432 0.1466 0.2570 EQ116 NDNS-Exact 0.2400 0.2327 0.3331 EQ116 NDNS-Partial 0.2123 0.2058 0.2946 EQ116 NDNS-Relaxed 0.2123 0.2058 0.2946 EQ117 NDNS-Exact 0.3245 0.3511 0.5322 EQ117 NDNS-Partial 0.3240 0.3715 0.6085 EQ117 NDNS-Relaxed 0.3240 0.3715 0.6085 EQ118 NDNS-Exact 0.6028 0.5646 0.6908 EQ118 NDNS-Partial 0.4655 0.4360 0.5335 EQ118 NDNS-Relaxed 0.4655 0.4360 0.5335 EQ119 NDNS-Exact 0.0797 0.2868 0.4027 EQ119 NDNS-Partial 0.0607 0.2263 0.3067 EQ119 NDNS-Relaxed 0.0607 0.2244 0.3067 EQ120 NDNS-Exact 0.2664 0.5027 0.7176 EQ120 NDNS-Partial 0.2512 0.4740 0.6767 EQ120 NDNS-Relaxed 0.2543 0.4798 0.6849 EQ121 NDNS-Exact 0.4126 0.3903 0.4689 EQ121 NDNS-Partial 0.4038 0.3819 0.4589 EQ121 NDNS-Relaxed 0.4038 0.3819 0.4589 EQ122 NDNS-Exact 0.2901 0.2465 0.3862 EQ122 NDNS-Partial 0.2421 0.2078 0.3409 EQ122 NDNS-Relaxed 0.2436 0.2092 0.3244 EQ123 NDNS-Exact 0.5281 0.5079 0.6317 EQ123 NDNS-Partial 0.4090 0.3934 0.4892 EQ123 NDNS-Relaxed 0.4100 0.3943 0.4904 EQ124 NDNS-Exact 0.2546 0.4694 0.6163 EQ124 NDNS-Partial 0.2189 0.4035 0.5298 EQ124 NDNS-Relaxed 0.2189 0.4035 0.5298 EQ125 NDNS-Exact 0.8139 0.3250 0.8570 EQ125 NDNS-Partial 0.6839 0.2731 0.7201 EQ125 NDNS-Relaxed 0.6839 0.2731 0.7201 EQ126 NDNS-Exact 0.1575 0.2966 0.6099 EQ126 NDNS-Partial 0.1272 0.2396 0.4927 EQ126 NDNS-Relaxed 0.1279 0.2408 0.4952 EQ127 NDNS-Exact 0.3200 0.4034 0.5870 EQ127 NDNS-Partial 0.2758 0.3684 0.5060 EQ127 NDNS-Relaxed 0.2813 0.3737 0.5160 EQ128 NDNS-Exact 0.1917 0.3416 0.4318 EQ128 NDNS-Partial 0.1439 0.2551 0.3393 EQ128 NDNS-Relaxed 0.1433 0.2555 0.3331 EQ129 NDNS-Exact 0.1296 0.1448 0.3360 EQ129 NDNS-Partial 0.1079 0.1205 0.2868 EQ129 NDNS-Relaxed 0.1079 0.1205 0.2868 EQ130 NDNS-Exact 0.5146 0.4404 0.6946 EQ130 NDNS-Partial 0.5146 0.4535 0.6946 EQ130 NDNS-Relaxed 0.5146 0.4535 0.6946 MEAN NDNS-Exact 0.3587 0.3802 0.4207 MEAN NDNS-Partial 0.3148 0.3377 0.3699 MEAN NDNS-Relaxed 0.3152 0.3387 0.3709 ******************************************************* Run: vigicovid_2 Query Metric Score Median Maximum EQ101 NDNS-Exact 0.2806 0.4974 0.6811 EQ101 NDNS-Partial 0.2323 0.4186 0.5638 EQ101 NDNS-Relaxed 0.2323 0.4186 0.5638 EQ102 NDNS-Exact 0.3630 0.3410 0.5418 EQ102 NDNS-Partial 0.3316 0.3115 0.4950 EQ102 NDNS-Relaxed 0.3316 0.3115 0.4950 EQ103 NDNS-Exact 0.2448 0.2500 0.3667 EQ103 NDNS-Partial 0.2035 0.2078 0.3048 EQ103 NDNS-Relaxed 0.2035 0.2078 0.3048 EQ104 NDNS-Exact 0.3782 0.4219 0.6273 EQ104 NDNS-Partial 0.3526 0.4080 0.6397 EQ104 NDNS-Relaxed 0.3526 0.4080 0.5977 EQ105 NDNS-Exact 0.4938 0.2766 0.7001 EQ105 NDNS-Partial 0.4668 0.2784 0.6617 EQ105 NDNS-Relaxed 0.4668 0.2784 0.6617 EQ106 NDNS-Exact 0.2505 0.4169 0.6451 EQ106 NDNS-Partial 0.2029 0.3377 0.5332 EQ106 NDNS-Relaxed 0.2029 0.3377 0.5225 EQ107 NDNS-Exact 0.1397 0.1240 0.1839 EQ107 NDNS-Partial 0.1288 0.1144 0.1695 EQ107 NDNS-Relaxed 0.1288 0.1144 0.1695 EQ108 NDNS-Exact 0.5957 0.3949 0.6287 EQ108 NDNS-Partial 0.5099 0.3380 0.5382 EQ108 NDNS-Relaxed 0.5099 0.3380 0.5382 EQ109 NDNS-Exact 0.5000 0.6309 1.0000 EQ109 NDNS-Partial 0.5000 0.7500 1.0000 EQ109 NDNS-Relaxed 0.5000 0.7500 1.0000 EQ110 NDNS-Exact 0.5424 0.3887 0.5470 EQ110 NDNS-Partial 0.4270 0.3022 0.4270 EQ110 NDNS-Relaxed 0.4175 0.2992 0.4210 EQ111 NDNS-Exact 0.4893 0.4480 0.5874 EQ111 NDNS-Partial 0.3838 0.3620 0.5094 EQ111 NDNS-Relaxed 0.3945 0.3688 0.5236 EQ112 NDNS-Exact 0.5765 0.5012 0.5953 EQ112 NDNS-Partial 0.4990 0.4338 0.5153 EQ112 NDNS-Relaxed 0.4990 0.4338 0.5153 EQ113 NDNS-Exact 0.4864 0.3610 0.5062 EQ113 NDNS-Partial 0.4730 0.3511 0.4923 EQ113 NDNS-Relaxed 0.4730 0.3511 0.4923 EQ114 NDNS-Exact 0.3260 0.3145 0.6719 EQ114 NDNS-Partial 0.3150 0.3039 0.6492 EQ114 NDNS-Relaxed 0.3150 0.3039 0.6492 EQ115 NDNS-Exact 0.1568 0.1533 0.2688 EQ115 NDNS-Partial 0.1494 0.1461 0.2561 EQ115 NDNS-Relaxed 0.1499 0.1466 0.2570 EQ116 NDNS-Exact 0.2403 0.2327 0.3331 EQ116 NDNS-Partial 0.2125 0.2058 0.2946 EQ116 NDNS-Relaxed 0.2125 0.2058 0.2946 EQ117 NDNS-Exact 0.3280 0.3511 0.5322 EQ117 NDNS-Partial 0.3274 0.3715 0.6085 EQ117 NDNS-Relaxed 0.3274 0.3715 0.6085 EQ118 NDNS-Exact 0.6091 0.5646 0.6908 EQ118 NDNS-Partial 0.4703 0.4360 0.5335 EQ118 NDNS-Relaxed 0.4703 0.4360 0.5335 EQ119 NDNS-Exact 0.0708 0.2868 0.4027 EQ119 NDNS-Partial 0.0539 0.2263 0.3067 EQ119 NDNS-Relaxed 0.0539 0.2244 0.3067 EQ120 NDNS-Exact 0.2626 0.5027 0.7176 EQ120 NDNS-Partial 0.2476 0.4740 0.6767 EQ120 NDNS-Relaxed 0.2506 0.4798 0.6849 EQ121 NDNS-Exact 0.3911 0.3903 0.4689 EQ121 NDNS-Partial 0.3828 0.3819 0.4589 EQ121 NDNS-Relaxed 0.3828 0.3819 0.4589 EQ122 NDNS-Exact 0.2953 0.2465 0.3862 EQ122 NDNS-Partial 0.2465 0.2078 0.3409 EQ122 NDNS-Relaxed 0.2481 0.2092 0.3244 EQ123 NDNS-Exact 0.5396 0.5079 0.6317 EQ123 NDNS-Partial 0.4179 0.3934 0.4892 EQ123 NDNS-Relaxed 0.4189 0.3943 0.4904 EQ124 NDNS-Exact 0.2579 0.4694 0.6163 EQ124 NDNS-Partial 0.2217 0.4035 0.5298 EQ124 NDNS-Relaxed 0.2217 0.4035 0.5298 EQ125 NDNS-Exact 0.8570 0.3250 0.8570 EQ125 NDNS-Partial 0.7201 0.2731 0.7201 EQ125 NDNS-Relaxed 0.7201 0.2731 0.7201 EQ126 NDNS-Exact 0.1621 0.2966 0.6099 EQ126 NDNS-Partial 0.1309 0.2396 0.4927 EQ126 NDNS-Relaxed 0.1316 0.2408 0.4952 EQ127 NDNS-Exact 0.3313 0.4034 0.5870 EQ127 NDNS-Partial 0.2856 0.3684 0.5060 EQ127 NDNS-Relaxed 0.2913 0.3737 0.5160 EQ128 NDNS-Exact 0.2006 0.3416 0.4318 EQ128 NDNS-Partial 0.1506 0.2551 0.3393 EQ128 NDNS-Relaxed 0.1500 0.2555 0.3331 EQ129 NDNS-Exact 0.1599 0.1448 0.3360 EQ129 NDNS-Partial 0.1331 0.1205 0.2868 EQ129 NDNS-Relaxed 0.1331 0.1205 0.2868 EQ130 NDNS-Exact 0.6946 0.4404 0.6946 EQ130 NDNS-Partial 0.6946 0.4535 0.6946 EQ130 NDNS-Relaxed 0.6946 0.4535 0.6946 MEAN NDNS-Exact 0.3741 0.3802 0.4207 MEAN NDNS-Partial 0.3290 0.3377 0.3699 MEAN NDNS-Relaxed 0.3295 0.3387 0.3709 ******************************************************* Run: vigicovid_3 Query Metric Score Median Maximum EQ101 NDNS-Exact 0.2783 0.4974 0.6811 EQ101 NDNS-Partial 0.2304 0.4186 0.5638 EQ101 NDNS-Relaxed 0.2304 0.4186 0.5638 EQ102 NDNS-Exact 0.3049 0.3410 0.5418 EQ102 NDNS-Partial 0.2786 0.3115 0.4950 EQ102 NDNS-Relaxed 0.2786 0.3115 0.4950 EQ103 NDNS-Exact 0.2421 0.2500 0.3667 EQ103 NDNS-Partial 0.2013 0.2078 0.3048 EQ103 NDNS-Relaxed 0.2013 0.2078 0.3048 EQ104 NDNS-Exact 0.4249 0.4219 0.6273 EQ104 NDNS-Partial 0.3961 0.4080 0.6397 EQ104 NDNS-Relaxed 0.3961 0.4080 0.5977 EQ105 NDNS-Exact 0.3790 0.2766 0.7001 EQ105 NDNS-Partial 0.3583 0.2784 0.6617 EQ105 NDNS-Relaxed 0.3583 0.2784 0.6617 EQ106 NDNS-Exact 0.2771 0.4169 0.6451 EQ106 NDNS-Partial 0.2244 0.3377 0.5332 EQ106 NDNS-Relaxed 0.2244 0.3377 0.5225 EQ107 NDNS-Exact 0.1552 0.1240 0.1839 EQ107 NDNS-Partial 0.1430 0.1144 0.1695 EQ107 NDNS-Relaxed 0.1430 0.1144 0.1695 EQ108 NDNS-Exact 0.6287 0.3949 0.6287 EQ108 NDNS-Partial 0.5382 0.3380 0.5382 EQ108 NDNS-Relaxed 0.5382 0.3380 0.5382 EQ109 NDNS-Exact 0.6309 0.6309 1.0000 EQ109 NDNS-Partial 0.6309 0.7500 1.0000 EQ109 NDNS-Relaxed 0.6309 0.7500 1.0000 EQ110 NDNS-Exact 0.5189 0.3887 0.5470 EQ110 NDNS-Partial 0.4105 0.3022 0.4270 EQ110 NDNS-Relaxed 0.3994 0.2992 0.4210 EQ111 NDNS-Exact 0.5226 0.4480 0.5874 EQ111 NDNS-Partial 0.4100 0.3620 0.5094 EQ111 NDNS-Relaxed 0.4214 0.3688 0.5236 EQ112 NDNS-Exact 0.5654 0.5012 0.5953 EQ112 NDNS-Partial 0.4894 0.4338 0.5153 EQ112 NDNS-Relaxed 0.4894 0.4338 0.5153 EQ113 NDNS-Exact 0.5062 0.3610 0.5062 EQ113 NDNS-Partial 0.4923 0.3511 0.4923 EQ113 NDNS-Relaxed 0.4923 0.3511 0.4923 EQ114 NDNS-Exact 0.3771 0.3145 0.6719 EQ114 NDNS-Partial 0.3644 0.3039 0.6492 EQ114 NDNS-Relaxed 0.3644 0.3039 0.6492 EQ115 NDNS-Exact 0.2049 0.1533 0.2688 EQ115 NDNS-Partial 0.1952 0.1461 0.2561 EQ115 NDNS-Relaxed 0.1959 0.1466 0.2570 EQ116 NDNS-Exact 0.2804 0.2327 0.3331 EQ116 NDNS-Partial 0.2480 0.2058 0.2946 EQ116 NDNS-Relaxed 0.2480 0.2058 0.2946 EQ117 NDNS-Exact 0.3513 0.3511 0.5322 EQ117 NDNS-Partial 0.3507 0.3715 0.6085 EQ117 NDNS-Relaxed 0.3507 0.3715 0.6085 EQ118 NDNS-Exact 0.6317 0.5646 0.6908 EQ118 NDNS-Partial 0.4878 0.4360 0.5335 EQ118 NDNS-Relaxed 0.4878 0.4360 0.5335 EQ119 NDNS-Exact 0.0816 0.2868 0.4027 EQ119 NDNS-Partial 0.0622 0.2263 0.3067 EQ119 NDNS-Relaxed 0.0622 0.2244 0.3067 EQ120 NDNS-Exact 0.3082 0.5027 0.7176 EQ120 NDNS-Partial 0.2906 0.4740 0.6767 EQ120 NDNS-Relaxed 0.2941 0.4798 0.6849 EQ121 NDNS-Exact 0.3894 0.3903 0.4689 EQ121 NDNS-Partial 0.3811 0.3819 0.4589 EQ121 NDNS-Relaxed 0.3811 0.3819 0.4589 EQ122 NDNS-Exact 0.3178 0.2465 0.3862 EQ122 NDNS-Partial 0.2652 0.2078 0.3409 EQ122 NDNS-Relaxed 0.2669 0.2092 0.3244 EQ123 NDNS-Exact 0.5194 0.5079 0.6317 EQ123 NDNS-Partial 0.4023 0.3934 0.4892 EQ123 NDNS-Relaxed 0.4032 0.3943 0.4904 EQ124 NDNS-Exact 0.4673 0.4694 0.6163 EQ124 NDNS-Partial 0.4017 0.4035 0.5298 EQ124 NDNS-Relaxed 0.4017 0.4035 0.5298 EQ125 NDNS-Exact 0.7061 0.3250 0.8570 EQ125 NDNS-Partial 0.5934 0.2731 0.7201 EQ125 NDNS-Relaxed 0.5934 0.2731 0.7201 EQ126 NDNS-Exact 0.1573 0.2966 0.6099 EQ126 NDNS-Partial 0.1271 0.2396 0.4927 EQ126 NDNS-Relaxed 0.1277 0.2408 0.4952 EQ127 NDNS-Exact 0.3561 0.4034 0.5870 EQ127 NDNS-Partial 0.3070 0.3684 0.5060 EQ127 NDNS-Relaxed 0.3131 0.3737 0.5160 EQ128 NDNS-Exact 0.2311 0.3416 0.4318 EQ128 NDNS-Partial 0.1734 0.2551 0.3393 EQ128 NDNS-Relaxed 0.1729 0.2555 0.3331 EQ129 NDNS-Exact 0.2912 0.1448 0.3360 EQ129 NDNS-Partial 0.2424 0.1205 0.2868 EQ129 NDNS-Relaxed 0.2424 0.1205 0.2868 EQ130 NDNS-Exact 0.6155 0.4404 0.6946 EQ130 NDNS-Partial 0.6155 0.4535 0.6946 EQ130 NDNS-Relaxed 0.6155 0.4535 0.6946 MEAN NDNS-Exact 0.3907 0.3802 0.4207 MEAN NDNS-Partial 0.3437 0.3377 0.3699 MEAN NDNS-Relaxed 0.3442 0.3387 0.3709 *******************************************************