==========================================================
TAC KBP 2017 CROSS-LINGUAL SLOT FILLING EVALUATION RESULTS
==========================================================


Team ID:  STANFORD
Organization:  Stanford University


*************************************************************

Run ID:  STANFORD_SF_XLING_1
Did the run access the live Web during the evaluation window:  No
Did the run extract relations from the Cold Start source corpus: Yes
Did the run generate meaningful confidence values: Yes
Run number of the English SF system that is most closely configured to the English component of this run: 1
Run number of the Spanish SF system that is most closely configured to the Spanish component of this run: 1
Run number of the Chinese SF system that is most closely configured to the Chinese component of this run: 1

Slot Filling Evaluation (Queries involve ONLY SF slots):

Scores based on P/R/F1; only k=1 justification allowed:
Metric                   RunID               Hop Prec   Recall F1    
SF-ALL-Micro             STANFORD_SF_XLING_1 0   0.4245 0.1288 0.1976
SF-ALL-Micro             STANFORD_SF_XLING_1 1   0.0830 0.0610 0.0703
SF-ALL-Micro             STANFORD_SF_XLING_1 ALL 0.2712 0.1117 0.1582
SF-ALL-Macro             STANFORD_SF_XLING_1 0   0.1812 0.1649 0.1529
SF-ALL-Macro             STANFORD_SF_XLING_1 1   0.0601 0.0596 0.0551
SF-ALL-Macro             STANFORD_SF_XLING_1 ALL 0.1222 0.1136 0.1053
LDC-MAX-ALL-Micro        STANFORD_SF_XLING_1 0   0.5740 0.3102 0.4028
LDC-MAX-ALL-Micro        STANFORD_SF_XLING_1 1   0.1649 0.1460 0.1549
LDC-MAX-ALL-Micro        STANFORD_SF_XLING_1 ALL 0.4288 0.2690 0.3306
LDC-MAX-ALL-Macro        STANFORD_SF_XLING_1 0   0.3940 0.3351 0.3363
LDC-MAX-ALL-Macro        STANFORD_SF_XLING_1 1   0.1353 0.1355 0.1274
LDC-MAX-ALL-Macro        STANFORD_SF_XLING_1 ALL 0.2688 0.2386 0.2352
LDC-MEAN-ALL-Macro       STANFORD_SF_XLING_1 0   0.1810 0.1652 0.1528
LDC-MEAN-ALL-Macro       STANFORD_SF_XLING_1 1   0.0601 0.0596 0.0551
LDC-MEAN-ALL-Macro       STANFORD_SF_XLING_1 ALL 0.1225 0.1141 0.1055

*ALL-Macro Prec, Recall and F1 refer to mean-precision, mean-recall and mean-F1.

Scores based on confidence values and Average Precision (AP); only k=1 justification allowed:
Metric                RunID               Hop AP    
SF-ALL-Macro          STANFORD_SF_XLING_1 0   0.1501
SF-ALL-Macro          STANFORD_SF_XLING_1 1   0.0185
SF-ALL-Macro          STANFORD_SF_XLING_1 ALL 0.0979
LDC-MEAN-ALL-Macro    STANFORD_SF_XLING_1 0   0.1498
LDC-MEAN-ALL-Macro    STANFORD_SF_XLING_1 1   0.0185
LDC-MEAN-ALL-Macro    STANFORD_SF_XLING_1 ALL 0.0978 (PRIMARY METRIC)

*ALL-Macro AP refer to mean of corresponding AP values.