========================================= TAC 2019 SM-KBP TASK 2 EVALUATION RESULTS ========================================= Team ID: ramfis Organization: University of Colorado Boulder; Brandeis University; Colorado State University ********************** Task 2 Zerohop Queries GT Sub Pooled Correct Dup Incrct Cntd Right Wrong Ignrd AP-B AP-W AP-T Run_ID 3178 8 8 6 0 2 8 6 2 0 0.0085 0.0085 0.0085 BBN_1.Colorado_1 3178 11 11 9 0 2 11 9 2 0 0.0092 0.0092 0.0092 GAIA_1-JHU_5.Colorado_2 3178 23 23 9 0 14 23 9 14 0 0.0369 0.0369 0.0369 GAIA_1-Michigan_1.Colorado_1 3178 35 35 29 0 6 35 29 6 0 0.0667 0.0667 0.0667 GAIA_1-OPERA_3.Colorado_1 3178 27 27 24 0 3 27 24 3 0 0.0627 0.0627 0.0627 GAIA_1.Colorado_2 3178 27 27 22 0 5 27 22 5 0 0.0612 0.0612 0.0612 GAIA_2.Colorado_1 3178 15 15 11 0 4 15 11 4 0 0.0128 0.0128 0.0128 OPERA_1.Colorado_2 3178 17 17 10 0 7 17 10 7 0 0.0123 0.0123 0.0123 OPERA_4.Colorado_1 ******************************************************************* Task 2 Graph Queries -- Strategy 1 (pool and evaluate single edges) GTA(1a) GT(1a) GTA(1b) GT(1b) Sub Pooled Correct PJLnkabl2O OLnkabl2QE Dup Incrct Cntd Right Wrong Ignrd Salient Prec(1a) Recall(1a) F1(1a) Recall(1b) Run_ID 680 472 176 175 93 63 45 34 21 7 18 48 14 34 15 2 0.2917 0.0297 0.0538 0.0114 BBN_1.Colorado_1 680 472 176 175 178 84 57 34 23 8 27 63 15 48 21 5 0.2381 0.0318 0.0561 0.0286 GAIA_1-JHU_5.Colorado_2 680 472 176 175 557 285 185 26 5 1 100 280 4 276 5 3 0.0143 0.0085 0.0106 0.0171 GAIA_1-Michigan_1.Colorado_1 680 472 176 175 482 311 233 127 93 24 78 226 69 157 85 16 0.3053 0.1462 0.1977 0.0914 GAIA_1-OPERA_3.Colorado_1 680 472 176 175 165 84 64 45 44 12 20 63 32 31 21 6 0.5079 0.0678 0.1196 0.0343 GAIA_1.Colorado_2 680 472 176 175 219 107 77 55 54 17 30 80 37 43 27 7 0.4625 0.0784 0.1341 0.0400 GAIA_2.Colorado_1 680 472 176 175 35 35 26 6 6 0 9 16 6 10 19 1 0.3750 0.0127 0.0246 0.0057 OPERA_1.Colorado_2 680 472 176 175 39 37 35 6 6 1 2 11 5 6 26 1 0.4545 0.0106 0.0207 0.0057 OPERA_4.Colorado_1