===========================================================
TAC KBP 2015 COLD START KB CONSTRUCTION EVALUATION RESULTS
===========================================================


Team ID:  SAFT_ISI
Organization:  USC Information Sciences Institute


*************************************************************

Run ID:  SAFT_ISI1
Did the run access the live Web during the evaluation window:  No
Run Number of most similar Cold Start Slot Filling task submission: NA
Did the run extract relations from the Cold Start source corpus: Yes
Did the run generate meaningful confidence values: Yes

Entity Discovery Evaluation:

Prec	Recall	F1	Metric
0.205	0.123	0.154	strong_mention_match
0.185	0.111	0.139	strong_typed_mention_match
0.000	0.000	0.000	entity_match
0.150	0.052	0.077	b_cubed
0.159	0.095	0.119	mention_ceaf
0.148	0.088	0.111	typed_mention_ceaf
0.148	0.088	0.111	typed_mention_ceaf_plus

Slot Filling Evaluation:

Metric		Hop	GoldT	Submit	Correct	Incorrect	Inexact	PIncorrect	Dup	Right	Wrong	Prec	Recall	F1
CSSF micro	0	4840	392	180	160		52	0		24	156	236	0.3980	0.0322	0.0596
CSSF micro	1	3954	401	78	124		28	171		8	70	331	0.1746	0.0177	0.0321
CSSF micro	ALL	8794	793	258	284		80	171		32	226	567	0.2850	0.0257	0.0471
LDC-MEAN macro	0														0.0656
LDC-MEAN macro	1														0.0160
LDC-MEAN macro	ALL														0.0478
LDC-MAX micro	0	1268	139	86	37		16	0		10	76	63	0.5468	0.0599	0.1080
LDC-MAX micro	1	900	125	36	48		8	33		3	33	92	0.2640	0.0367	0.0644
LDC-MAX micro	ALL	2168	264	122	85		24	33		13	109	155	0.4129	0.0503	0.0896
LDC-MAX macro	0														0.1004
LDC-MAX macro	1														0.0204
LDC-MAX macro	ALL														0.0718