==================================================== TAC KBP 2016 CHINESE SLOT FILLING EVALUATION RESULTS ==================================================== Team ID: Stanford Organization: Stanford University ************************************************************* Run ID: Stanford_SF_CMN_1 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro Stanford_SF_CMN_1 0 0.6178 0.1292 0.2137 SF-ALL-Micro Stanford_SF_CMN_1 1 0.2128 0.0435 0.0722 SF-ALL-Micro Stanford_SF_CMN_1 ALL 0.5245 0.1091 0.1806 SF-ALL-Macro Stanford_SF_CMN_1 0 0.1550 0.1279 0.1321 SF-ALL-Macro Stanford_SF_CMN_1 1 0.0530 0.0662 0.0574 SF-ALL-Macro Stanford_SF_CMN_1 ALL 0.1077 0.0993 0.0975 LDC-MAX-ALL-Micro Stanford_SF_CMN_1 0 0.5818 0.1196 0.1984 LDC-MAX-ALL-Micro Stanford_SF_CMN_1 1 0.1538 0.0326 0.0538 LDC-MAX-ALL-Micro Stanford_SF_CMN_1 ALL 0.4698 0.0974 0.1613 LDC-MAX-ALL-Macro Stanford_SF_CMN_1 0 0.1621 0.1389 0.1407 LDC-MAX-ALL-Macro Stanford_SF_CMN_1 1 0.0455 0.0545 0.0485 LDC-MAX-ALL-Macro Stanford_SF_CMN_1 ALL 0.1095 0.1009 0.0991 LDC-MEAN-ALL-Macro Stanford_SF_CMN_1 0 0.1534 0.1315 0.1329 LDC-MEAN-ALL-Macro Stanford_SF_CMN_1 1 0.0455 0.0545 0.0485 LDC-MEAN-ALL-Macro Stanford_SF_CMN_1 ALL 0.1047 0.0968 0.0949 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.3085 0.9756 0.4688 ************************************************************* Run ID: Stanford_SF_CMN_2 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro Stanford_SF_CMN_2 0 0.1376 0.1798 0.1559 SF-ALL-Micro Stanford_SF_CMN_2 1 0.0254 0.0783 0.0383 SF-ALL-Micro Stanford_SF_CMN_2 ALL 0.0905 0.1560 0.1146 SF-ALL-Macro Stanford_SF_CMN_2 0 0.1672 0.1955 0.1539 SF-ALL-Macro Stanford_SF_CMN_2 1 0.0523 0.0927 0.0615 SF-ALL-Macro Stanford_SF_CMN_2 ALL 0.1140 0.1479 0.1111 LDC-MAX-ALL-Micro Stanford_SF_CMN_2 0 0.1352 0.2019 0.1619 LDC-MAX-ALL-Micro Stanford_SF_CMN_2 1 0.0218 0.0761 0.0339 LDC-MAX-ALL-Micro Stanford_SF_CMN_2 ALL 0.0847 0.1697 0.1130 LDC-MAX-ALL-Macro Stanford_SF_CMN_2 0 0.1730 0.2198 0.1654 LDC-MAX-ALL-Macro Stanford_SF_CMN_2 1 0.0669 0.1091 0.0767 LDC-MAX-ALL-Macro Stanford_SF_CMN_2 ALL 0.1252 0.1699 0.1254 LDC-MEAN-ALL-Macro Stanford_SF_CMN_2 0 0.1689 0.2160 0.1623 LDC-MEAN-ALL-Macro Stanford_SF_CMN_2 1 0.0669 0.1091 0.0767 LDC-MEAN-ALL-Macro Stanford_SF_CMN_2 ALL 0.1229 0.1678 0.1237 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.2902 0.8943 0.4382 ************************************************************* Run ID: Stanford_SF_CMN_3 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro Stanford_SF_CMN_3 0 0.5944 0.1132 0.1902 SF-ALL-Micro Stanford_SF_CMN_3 1 0.2128 0.0435 0.0722 SF-ALL-Micro Stanford_SF_CMN_3 ALL 0.5000 0.0968 0.1623 SF-ALL-Macro Stanford_SF_CMN_3 0 0.1407 0.1122 0.1169 SF-ALL-Macro Stanford_SF_CMN_3 1 0.0530 0.0662 0.0574 SF-ALL-Macro Stanford_SF_CMN_3 ALL 0.1001 0.0909 0.0894 LDC-MAX-ALL-Micro Stanford_SF_CMN_3 0 0.5686 0.1084 0.1821 LDC-MAX-ALL-Micro Stanford_SF_CMN_3 1 0.1538 0.0326 0.0538 LDC-MAX-ALL-Micro Stanford_SF_CMN_3 ALL 0.4539 0.0890 0.1488 LDC-MAX-ALL-Macro Stanford_SF_CMN_3 0 0.1509 0.1240 0.1273 LDC-MAX-ALL-Macro Stanford_SF_CMN_3 1 0.0455 0.0545 0.0485 LDC-MAX-ALL-Macro Stanford_SF_CMN_3 ALL 0.1033 0.0927 0.0918 LDC-MEAN-ALL-Macro Stanford_SF_CMN_3 0 0.1422 0.1175 0.1206 LDC-MEAN-ALL-Macro Stanford_SF_CMN_3 1 0.0455 0.0545 0.0485 LDC-MEAN-ALL-Macro Stanford_SF_CMN_3 ALL 0.0986 0.0891 0.0881 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.3085 0.9756 0.4688 ************************************************************* Run ID: Stanford_SF_CMN_4 Did the run access the live Web during the evaluation window: No Did the run extract relations from the Cold Start source corpus: Yes Did the run generate meaningful confidence values: Yes Slot Filling Evaluation: Metric RunID Hop Prec Recall F1 SF-ALL-Micro Stanford_SF_CMN_4 0 0.5890 0.1145 0.1918 SF-ALL-Micro Stanford_SF_CMN_4 1 0.2128 0.0435 0.0722 SF-ALL-Micro Stanford_SF_CMN_4 ALL 0.4974 0.0979 0.1635 SF-ALL-Macro Stanford_SF_CMN_4 0 0.1435 0.1179 0.1207 SF-ALL-Macro Stanford_SF_CMN_4 1 0.0530 0.0662 0.0574 SF-ALL-Macro Stanford_SF_CMN_4 ALL 0.1016 0.0940 0.0914 LDC-MAX-ALL-Micro Stanford_SF_CMN_4 0 0.5619 0.1103 0.1844 LDC-MAX-ALL-Micro Stanford_SF_CMN_4 1 0.1538 0.0326 0.0538 LDC-MAX-ALL-Micro Stanford_SF_CMN_4 ALL 0.4514 0.0904 0.1506 LDC-MAX-ALL-Macro Stanford_SF_CMN_4 0 0.1546 0.1315 0.1323 LDC-MAX-ALL-Macro Stanford_SF_CMN_4 1 0.0455 0.0545 0.0485 LDC-MAX-ALL-Macro Stanford_SF_CMN_4 ALL 0.1054 0.0968 0.0945 LDC-MEAN-ALL-Macro Stanford_SF_CMN_4 0 0.1459 0.1249 0.1255 LDC-MEAN-ALL-Macro Stanford_SF_CMN_4 1 0.0455 0.0545 0.0485 LDC-MEAN-ALL-Macro Stanford_SF_CMN_4 ALL 0.1006 0.0932 0.0908 *ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1. NIL-DETECTION P/R/F1: 0.3085 0.9756 0.4688