=========================================================================
TAC KBP 2016 SLOT FILLER VALIDATION SPANISH ENSEMBLING EVALUATION RESULTS
=========================================================================


Team ID:  SAFT_ISI
Organization:  USC Information Sciences Institute


*************************************************************

Run ID:  SAFT_ISI1
Did the run access the live Web during the evaluation window:  No
Did this run judge each candidate slot filler independently of all other candidate slot fillers in the evaluation dataset: Yes
Did this run judge candidate slot fillers for each slot-filling run independently of all other slot-filling runs in the evaluation dataset: No
Did this run judge candidate slot fillers for each slot-filling team independently of all other slot-filling teams in the evaluation dataset: No
Did this run make use of the slot filler or justification offsets provided for each candidate slot filler: Yes
Did this run make use of the confidence values provided for each candidate slot filler: No
Did this run make use of the system profiles for the slot filling runs: No
Did this run make use of the preliminary assessments provided for some of the slot filler candidates: Yes

CSSF micro-average Precision, Recall, and F1, at each hop level:
						hop0_P	hop0_R	hop0_F		hop1_P	hop0_R	hop1_F		ALL_P	ALL_R	ALL_F
SAFT_ISI1.SPA.ensemble				0.2391	0.0374	0.0647		0.0000	0.0000	0.0000		0.2391	0.0220	0.0402
UMass_IESL_SF_SPA_4 (best hop0 F1 input)	0.1458	0.2653	0.1882		0.0000	0.0000	0.0000		0.1458	0.1557	0.1506
UMass_IESL_KB_SPA_3 (best hop1 F1 input)	0.2991	0.1190	0.1703		0.0513	0.0290	0.0370		0.1752	0.0818	0.1116
UMass_IESL_SF_SPA_4 (best ALL  F1 input)	0.1458	0.2653	0.1882		0.0000	0.0000	0.0000		0.1458	0.1557	0.1506

MAX micro-average Precision, Recall, and F1, at each hop level:
						hop0_P	hop0_R	hop0_F		hop1_P	hop0_R	hop1_F		ALL_P	ALL_R	ALL_F
SAFT_ISI1.SPA.ensemble				0.2162	0.0367	0.0627		0.0000	0.0000	0.0000		0.2162	0.0237	0.0427
UMass_IESL_SF_SPA_4 (best hop0 F1 input)	0.1335	0.2615	0.1767		0.0000	0.0000	0.0000		0.1335	0.1686	0.1490
UMass_IESL_KB_SPA_3 (best hop1 F1 input)	0.2842	0.1239	0.1725		0.0600	0.0500	0.0545		0.1692	0.0976	0.1238
UMass_IESL_SF_SPA_4 (best ALL  F1 input)	0.1335	0.2615	0.1767		0.0000	0.0000	0.0000		0.1335	0.1686	0.1490

MEAN macro-average Precision, Recall, and F1, at each hop level:
						hop0_P	hop0_R	hop0_F		hop1_P	hop0_R	hop1_F		ALL_P	ALL_R	ALL_F
SAFT_ISI1.SPA.ensemble				0.0385	0.0470	0.0414		0.0000	0.0000	0.0000		0.0254	0.0310	0.0273
UMass_IESL_SF_SPA_4 (best hop0 F1 input)	0.1652	0.2610	0.1815		0.0000	0.0000	0.0000		0.1089	0.1721	0.1196
UMass_IESL_KB_SPA_2 (best hop1 F1 input)	0.1031	0.1414	0.1031		0.0168	0.0316	0.0196		0.0737	0.1040	0.0746
UMass_IESL_SF_SPA_4 (best ALL  F1 input)	0.1652	0.2610	0.1815		0.0000	0.0000	0.0000		0.1089	0.1721	0.1196


*************************************************************

Run ID:  SAFT_ISI2
Did the run access the live Web during the evaluation window:  No
Did this run judge each candidate slot filler independently of all other candidate slot fillers in the evaluation dataset: Yes
Did this run judge candidate slot fillers for each slot-filling run independently of all other slot-filling runs in the evaluation dataset: No
Did this run judge candidate slot fillers for each slot-filling team independently of all other slot-filling teams in the evaluation dataset: No
Did this run make use of the slot filler or justification offsets provided for each candidate slot filler: Yes
Did this run make use of the confidence values provided for each candidate slot filler: No
Did this run make use of the system profiles for the slot filling runs: No
Did this run make use of the preliminary assessments provided for some of the slot filler candidates: Yes

CSSF micro-average Precision, Recall, and F1, at each hop level:
						hop0_P	hop0_R	hop0_F		hop1_P	hop0_R	hop1_F		ALL_P	ALL_R	ALL_F
SAFT_ISI2.SPA.ensemble				0.2391	0.0374	0.0647		0.0000	0.0000	0.0000		0.2391	0.0220	0.0402
UMass_IESL_SF_SPA_4 (best hop0 F1 input)	0.1458	0.2653	0.1882		0.0000	0.0000	0.0000		0.1458	0.1557	0.1506
UMass_IESL_KB_SPA_3 (best hop1 F1 input)	0.2991	0.1190	0.1703		0.0513	0.0290	0.0370		0.1752	0.0818	0.1116
UMass_IESL_SF_SPA_4 (best ALL  F1 input)	0.1458	0.2653	0.1882		0.0000	0.0000	0.0000		0.1458	0.1557	0.1506

MAX micro-average Precision, Recall, and F1, at each hop level:
						hop0_P	hop0_R	hop0_F		hop1_P	hop0_R	hop1_F		ALL_P	ALL_R	ALL_F
SAFT_ISI2.SPA.ensemble				0.2162	0.0367	0.0627		0.0000	0.0000	0.0000		0.2162	0.0237	0.0427
UMass_IESL_SF_SPA_4 (best hop0 F1 input)	0.1335	0.2615	0.1767		0.0000	0.0000	0.0000		0.1335	0.1686	0.1490
UMass_IESL_KB_SPA_3 (best hop1 F1 input)	0.2842	0.1239	0.1725		0.0600	0.0500	0.0545		0.1692	0.0976	0.1238
UMass_IESL_SF_SPA_4 (best ALL  F1 input)	0.1335	0.2615	0.1767		0.0000	0.0000	0.0000		0.1335	0.1686	0.1490

MEAN macro-average Precision, Recall, and F1, at each hop level:
						hop0_P	hop0_R	hop0_F		hop1_P	hop0_R	hop1_F		ALL_P	ALL_R	ALL_F
SAFT_ISI2.SPA.ensemble				0.0385	0.0470	0.0414		0.0000	0.0000	0.0000		0.0254	0.0310	0.0273
UMass_IESL_SF_SPA_4 (best hop0 F1 input)	0.1652	0.2610	0.1815		0.0000	0.0000	0.0000		0.1089	0.1721	0.1196
UMass_IESL_KB_SPA_2 (best hop1 F1 input)	0.1031	0.1414	0.1031		0.0168	0.0316	0.0196		0.0737	0.1040	0.0746
UMass_IESL_SF_SPA_4 (best ALL  F1 input)	0.1652	0.2610	0.1815		0.0000	0.0000	0.0000		0.1089	0.1721	0.1196