========================================= TAC 2019 SM-KBP TASK 1 EVALUATION RESULTS ========================================= Team ID: OPERA Organization: Carnegie Mellon University; USC Information Sciences Institute (ISI) TA1b_OPERA_TA1b_aditi_V5 (for Task 1b) is comparable to TA1a_OPERA_TA1a_aditi_V5 (for Task 1a), and operates in the context of one of the what-if hypotheses (for E101, E102, or E103) ******************** Task 1 Class Queries GT Sub Pooled Correct Dup Incrct Cntd Right Wrong Ignrd AP-B AP-W AP-T Run_ID 1126 2023 2023 1735 1276 288 2023 459 1564 0 0.4527 0.3697 0.4020 TA1a_OPERA_TA1a_aditi_V2 1126 1215 1215 1054 609 161 1215 445 770 0 0.4354 0.3483 0.3844 TA1a_OPERA_TA1a_aditi_V4 1126 1161 1161 1005 575 156 1161 430 731 0 0.4297 0.3518 0.3815 TA1a_OPERA_TA1a_aditi_V5 1126 2074 2074 1780 1313 294 2074 467 1607 0 0.4518 0.3814 0.3920 TA1a_OPERA_TA1a_hans_V2 1126 1281 1281 1122 665 159 1281 457 824 0 0.4451 0.3698 0.3818 TA1a_OPERA_TA1a_hans_V3 1126 1163 1163 1004 575 159 1163 429 734 0 0.4317 0.3535 0.3823 TA1b_OPERA_TA1b_aditi_V5-E101_PT003 1126 1162 1162 1005 575 157 1162 430 732 0 0.4303 0.3520 0.3850 TA1b_OPERA_TA1b_aditi_V5-E102_PT002 1126 1165 1165 1006 574 159 1165 432 733 0 0.4328 0.3546 0.3862 TA1b_OPERA_TA1b_aditi_V5-E103_PT001 ******************** Task 1 Graph Queries GTA GT Sub Pooled Correct PJLnkabl2O Dup Incrct Cntd Right Wrong Ignrd Prec Recall F1 Run_ID 994 994 175 175 96 45 0 79 152 45 107 23 0.2961 0.0453 0.0785 TA1a_OPERA_TA1a_aditi_V2 994 994 175 175 96 45 0 79 152 45 107 23 0.2961 0.0453 0.0785 TA1a_OPERA_TA1a_aditi_V4 994 994 174 174 94 39 0 80 153 39 114 21 0.2549 0.0392 0.0680 TA1a_OPERA_TA1a_aditi_V5 994 994 159 159 87 47 0 72 139 47 92 20 0.3381 0.0473 0.0830 TA1a_OPERA_TA1a_hans_V2 994 994 167 167 94 53 0 73 147 53 94 20 0.3605 0.0533 0.0929 TA1a_OPERA_TA1a_hans_V3 994 994 174 174 94 37 0 80 153 37 116 21 0.2418 0.0372 0.0645 TA1b_OPERA_TA1b_aditi_V5-E101_PT003 994 994 174 174 95 36 0 79 155 36 119 19 0.2323 0.0362 0.0627 TA1b_OPERA_TA1b_aditi_V5-E102_PT002 994 994 173 173 93 37 0 80 156 37 119 17 0.2372 0.0372 0.0643 TA1b_OPERA_TA1b_aditi_V5-E103_PT001