======================================================================= TAC KBP 2015 TRILINGUAL ENTITY DISCOVERY AND LINKING EVALUATION RESULTS ======================================================================= Team ID: RPI_BLENDER Organization: Rensselear Polytechnic Institute ************************************************************* Run ID: RPI_BLENDER1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.791 0.536 0.639 strong_mention_match 0.714 0.484 0.577 strong_typed_mention_match 0.599 0.406 0.484 strong_typed_all_match 0.578 0.380 0.458 strong_typed_link_match 0.655 0.479 0.554 strong_typed_nil_match 0.589 0.478 0.528 entity_match 0.683 0.462 0.551 mention_ceaf 0.641 0.434 0.517 typed_mention_ceaf 0.595 0.403 0.480 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.818 0.649 0.724 strong_mention_match 0.746 0.591 0.659 strong_typed_mention_match 0.629 0.498 0.556 strong_typed_all_match 0.607 0.530 0.566 strong_typed_link_match 0.710 0.420 0.527 strong_typed_nil_match 0.626 0.647 0.636 entity_match 0.737 0.584 0.652 mention_ceaf 0.682 0.541 0.603 typed_mention_ceaf 0.626 0.496 0.553 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.794 0.389 0.522 strong_mention_match 0.701 0.343 0.460 strong_typed_mention_match 0.579 0.283 0.380 strong_typed_all_match 0.539 0.210 0.302 strong_typed_link_match 0.644 0.547 0.591 strong_typed_nil_match 0.576 0.285 0.381 entity_match 0.680 0.333 0.447 mention_ceaf 0.648 0.317 0.426 typed_mention_ceaf 0.577 0.282 0.379 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.710 0.521 0.601 strong_mention_match 0.642 0.471 0.543 strong_typed_mention_match 0.542 0.397 0.459 strong_typed_all_match 0.522 0.342 0.414 strong_typed_link_match 0.578 0.539 0.558 strong_typed_nil_match 0.512 0.410 0.455 entity_match 0.630 0.462 0.533 mention_ceaf 0.585 0.429 0.495 typed_mention_ceaf 0.534 0.391 0.452 typed_mention_ceaf_plus ************************************************************* Run ID: RPI_BLENDER2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.791 0.536 0.639 strong_mention_match 0.714 0.483 0.576 strong_typed_mention_match 0.594 0.402 0.480 strong_typed_all_match 0.571 0.372 0.450 strong_typed_link_match 0.651 0.487 0.557 strong_typed_nil_match 0.589 0.472 0.524 entity_match 0.672 0.455 0.542 mention_ceaf 0.631 0.427 0.510 typed_mention_ceaf 0.587 0.398 0.474 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.818 0.649 0.724 strong_mention_match 0.745 0.591 0.659 strong_typed_mention_match 0.619 0.490 0.547 strong_typed_all_match 0.596 0.513 0.551 strong_typed_link_match 0.698 0.435 0.536 strong_typed_nil_match 0.627 0.633 0.630 entity_match 0.716 0.567 0.633 mention_ceaf 0.665 0.527 0.588 typed_mention_ceaf 0.612 0.485 0.541 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.794 0.389 0.522 strong_mention_match 0.701 0.343 0.460 strong_typed_mention_match 0.579 0.283 0.380 strong_typed_all_match 0.539 0.210 0.302 strong_typed_link_match 0.644 0.547 0.591 strong_typed_nil_match 0.576 0.285 0.381 entity_match 0.680 0.333 0.447 mention_ceaf 0.648 0.317 0.426 typed_mention_ceaf 0.577 0.282 0.379 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.710 0.521 0.601 strong_mention_match 0.642 0.471 0.543 strong_typed_mention_match 0.542 0.397 0.459 strong_typed_all_match 0.522 0.342 0.414 strong_typed_link_match 0.578 0.539 0.558 strong_typed_nil_match 0.512 0.410 0.455 entity_match 0.630 0.462 0.533 mention_ceaf 0.585 0.429 0.495 typed_mention_ceaf 0.534 0.391 0.452 typed_mention_ceaf_plus ************************************************************* Run ID: RPI_BLENDER3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.767 0.553 0.643 strong_mention_match 0.695 0.500 0.582 strong_typed_mention_match 0.575 0.414 0.482 strong_typed_all_match 0.578 0.380 0.458 strong_typed_link_match 0.571 0.512 0.540 strong_typed_nil_match 0.589 0.478 0.528 entity_match 0.633 0.456 0.530 mention_ceaf 0.596 0.429 0.499 typed_mention_ceaf 0.554 0.399 0.464 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.775 0.684 0.727 strong_mention_match 0.709 0.626 0.665 strong_typed_mention_match 0.585 0.516 0.548 strong_typed_all_match 0.607 0.530 0.566 strong_typed_link_match 0.531 0.482 0.505 strong_typed_nil_match 0.626 0.647 0.636 entity_match 0.668 0.590 0.627 mention_ceaf 0.621 0.548 0.582 typed_mention_ceaf 0.570 0.503 0.535 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.794 0.389 0.522 strong_mention_match 0.701 0.343 0.460 strong_typed_mention_match 0.579 0.283 0.380 strong_typed_all_match 0.539 0.210 0.302 strong_typed_link_match 0.644 0.547 0.591 strong_typed_nil_match 0.576 0.285 0.381 entity_match 0.680 0.333 0.447 mention_ceaf 0.648 0.317 0.426 typed_mention_ceaf 0.577 0.282 0.379 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.710 0.521 0.601 strong_mention_match 0.642 0.471 0.543 strong_typed_mention_match 0.542 0.397 0.459 strong_typed_all_match 0.522 0.342 0.414 strong_typed_link_match 0.578 0.539 0.558 strong_typed_nil_match 0.512 0.410 0.455 entity_match 0.630 0.462 0.533 mention_ceaf 0.585 0.429 0.495 typed_mention_ceaf 0.534 0.391 0.452 typed_mention_ceaf_plus ************************************************************* Run ID: RPI_BLENDER4 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: Yes ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.767 0.553 0.643 strong_mention_match 0.694 0.500 0.581 strong_typed_mention_match 0.570 0.411 0.477 strong_typed_all_match 0.571 0.372 0.450 strong_typed_link_match 0.569 0.520 0.543 strong_typed_nil_match 0.589 0.472 0.524 entity_match 0.622 0.448 0.521 mention_ceaf 0.587 0.423 0.492 typed_mention_ceaf 0.546 0.394 0.458 typed_mention_ceaf_plus ONLY English documents: Prec Recall F1 Metric 0.775 0.684 0.727 strong_mention_match 0.708 0.625 0.664 strong_typed_mention_match 0.576 0.508 0.540 strong_typed_all_match 0.596 0.513 0.551 strong_typed_link_match 0.529 0.497 0.513 strong_typed_nil_match 0.627 0.633 0.630 entity_match 0.649 0.573 0.609 mention_ceaf 0.605 0.534 0.568 typed_mention_ceaf 0.557 0.492 0.523 typed_mention_ceaf_plus ONLY Chinese documents: Prec Recall F1 Metric 0.794 0.389 0.522 strong_mention_match 0.701 0.343 0.460 strong_typed_mention_match 0.579 0.283 0.380 strong_typed_all_match 0.539 0.210 0.302 strong_typed_link_match 0.644 0.547 0.591 strong_typed_nil_match 0.576 0.285 0.381 entity_match 0.680 0.333 0.447 mention_ceaf 0.648 0.317 0.426 typed_mention_ceaf 0.577 0.282 0.379 typed_mention_ceaf_plus ONLY Spanish documents: Prec Recall F1 Metric 0.710 0.521 0.601 strong_mention_match 0.642 0.471 0.543 strong_typed_mention_match 0.542 0.397 0.459 strong_typed_all_match 0.522 0.342 0.414 strong_typed_link_match 0.578 0.539 0.558 strong_typed_nil_match 0.512 0.410 0.455 entity_match 0.630 0.462 0.533 mention_ceaf 0.585 0.429 0.495 typed_mention_ceaf 0.534 0.391 0.452 typed_mention_ceaf_plus