======================================================= TAC KBP 2015 COLD START SLOT FILLING EVALUATION RESULTS ======================================================= Team ID: UGENT_IBCN Organization: Ghent University - iMinds ************************************************************* Run ID: UGENT_IBCN1 Did the run access the live Web during the evaluation window: No Run Number of most similar Cold Start KB Construction task submission: NA Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: No Slot Filling Evaluation: Metric Hop GoldT Submit Correct Incorrect Inexact PIncorrect Dup Right Wrong Prec Recall F1 CSSF micro 0 4840 3257 1121 2002 134 0 41 1080 2177 0.3316 0.2231 0.2668 CSSF micro 1 3954 1874 325 491 68 990 18 307 1567 0.1638 0.0776 0.1054 CSSF micro ALL 8794 5131 1446 2493 202 990 59 1387 3744 0.2703 0.1577 0.1992 LDC-MEAN macro 0 0.3079 LDC-MEAN macro 1 0.1175 LDC-MEAN macro ALL 0.2397 LDC-MAX micro 0 1268 903 330 531 42 0 13 317 586 0.3511 0.2500 0.2920 LDC-MAX micro 1 900 511 87 125 20 279 4 83 428 0.1624 0.0922 0.1176 LDC-MAX micro ALL 2168 1414 417 656 62 279 17 400 1014 0.2829 0.1845 0.2233 LDC-MAX macro 0 0.3429 LDC-MAX macro 1 0.1349 LDC-MAX macro ALL 0.2684 ************************************************************* Run ID: UGENT_IBCN2 Did the run access the live Web during the evaluation window: No Run Number of most similar Cold Start KB Construction task submission: NA Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: No Slot Filling Evaluation: Metric Hop GoldT Submit Correct Incorrect Inexact PIncorrect Dup Right Wrong Prec Recall F1 CSSF micro 0 4840 3257 1121 2002 134 0 41 1080 2177 0.3316 0.2231 0.2668 CSSF micro 1 3954 1921 321 510 63 1027 18 303 1618 0.1577 0.0766 0.1031 CSSF micro ALL 8794 5178 1442 2512 197 1027 59 1383 3795 0.2671 0.1573 0.1980 LDC-MEAN macro 0 0.3079 LDC-MEAN macro 1 0.1127 LDC-MEAN macro ALL 0.2380 LDC-MAX micro 0 1268 906 330 535 41 0 13 317 589 0.3499 0.2500 0.2916 LDC-MAX micro 1 900 500 87 131 19 263 4 83 417 0.1660 0.0922 0.1186 LDC-MAX micro ALL 2168 1406 417 666 60 263 17 400 1006 0.2845 0.1845 0.2238 LDC-MAX macro 0 0.3429 LDC-MAX macro 1 0.1310 LDC-MAX macro ALL 0.2670 ************************************************************* Run ID: UGENT_IBCN3 Did the run access the live Web during the evaluation window: No Run Number of most similar Cold Start KB Construction task submission: NA Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: No Slot Filling Evaluation: Metric Hop GoldT Submit Correct Incorrect Inexact PIncorrect Dup Right Wrong Prec Recall F1 CSSF micro 0 4840 3455 1132 2178 145 0 45 1087 2368 0.3146 0.2246 0.2621 CSSF micro 1 3954 2339 327 635 68 1309 18 309 2030 0.1321 0.0781 0.0982 CSSF micro ALL 8794 5794 1459 2813 213 1309 63 1396 4398 0.2409 0.1587 0.1914 LDC-MEAN macro 0 0.3049 LDC-MEAN macro 1 0.1199 LDC-MEAN macro ALL 0.2387 LDC-MAX micro 0 1268 959 333 580 46 0 14 319 640 0.3326 0.2516 0.2865 LDC-MAX micro 1 900 614 89 162 21 342 4 85 529 0.1384 0.0944 0.1123 LDC-MAX micro ALL 2168 1573 422 742 67 342 18 404 1169 0.2568 0.1863 0.2160 LDC-MAX macro 0 0.3399 LDC-MAX macro 1 0.1377 LDC-MAX macro ALL 0.2675 ************************************************************* Run ID: UGENT_IBCN4 Did the run access the live Web during the evaluation window: No Run Number of most similar Cold Start KB Construction task submission: NA Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: No Slot Filling Evaluation: Metric Hop GoldT Submit Correct Incorrect Inexact PIncorrect Dup Right Wrong Prec Recall F1 CSSF micro 0 4840 4562 1262 3107 193 0 69 1193 3369 0.2615 0.2465 0.2538 CSSF micro 1 3954 2717 406 807 87 1417 18 388 2329 0.1428 0.0981 0.1163 CSSF micro ALL 8794 7279 1668 3914 280 1417 87 1581 5698 0.2172 0.1798 0.1967 LDC-MEAN macro 0 0.3047 LDC-MEAN macro 1 0.1164 LDC-MEAN macro ALL 0.2373 LDC-MAX micro 0 1268 1246 368 818 60 0 20 348 898 0.2793 0.2744 0.2768 LDC-MAX micro 1 900 745 113 209 25 398 4 109 636 0.1463 0.1211 0.1325 LDC-MAX micro ALL 2168 1991 481 1027 85 398 24 457 1534 0.2295 0.2108 0.2198 LDC-MAX macro 0 0.3397 LDC-MAX macro 1 0.1345 LDC-MAX macro ALL 0.2663