=============================================================================== TAC KBP 2016 ENTITY DISCOVERY AND LINKING EVALUATION RESULTS (FIRST EDL WINDOW) =============================================================================== Team ID: RPI_BLENDER Organization: Rensselaer Polytechnic Institute ************************************************************* Run ID: RPI_BLENDER1 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.832 0.484 0.612 strong_typed_mention_match 0.764 0.445 0.563 strong_typed_all_match 0.754 0.417 0.537 strong_typed_link_match 0.792 0.545 0.646 strong_typed_nil_match 0.710 0.563 0.628 entity_match 0.807 0.470 0.594 mention_ceaf 0.773 0.450 0.569 typed_mention_ceaf 0.740 0.431 0.545 typed_mention_ceaf_plus 0.808 0.332 0.471 b_cubed 0.733 0.311 0.437 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.935 0.574 0.711 strong_mention_match 0.884 0.543 0.673 strong_typed_mention_match 0.816 0.501 0.621 strong_typed_all_match 0.789 0.493 0.607 strong_typed_link_match 0.919 0.530 0.672 strong_typed_nil_match 0.761 0.651 0.702 entity_match 0.876 0.537 0.666 mention_ceaf 0.833 0.511 0.633 typed_mention_ceaf 0.794 0.487 0.604 typed_mention_ceaf_plus 0.884 0.406 0.557 b_cubed 0.809 0.382 0.519 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.855 0.454 0.593 strong_mention_match 0.820 0.435 0.568 strong_typed_mention_match 0.761 0.404 0.527 strong_typed_all_match 0.747 0.383 0.507 strong_typed_link_match 0.808 0.487 0.608 strong_typed_nil_match 0.683 0.499 0.577 entity_match 0.822 0.436 0.570 mention_ceaf 0.793 0.420 0.549 typed_mention_ceaf 0.751 0.398 0.520 typed_mention_ceaf_plus 0.787 0.313 0.448 b_cubed 0.725 0.291 0.415 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.824 0.500 0.623 strong_mention_match 0.774 0.470 0.585 strong_typed_mention_match 0.699 0.424 0.528 strong_typed_all_match 0.705 0.356 0.473 strong_typed_link_match 0.691 0.617 0.652 strong_typed_nil_match 0.668 0.523 0.587 entity_match 0.725 0.440 0.547 mention_ceaf 0.694 0.421 0.524 typed_mention_ceaf 0.656 0.398 0.496 typed_mention_ceaf_plus 0.750 0.326 0.455 b_cubed 0.656 0.300 0.411 b_cubed_plus ************************************************************* Run ID: RPI_BLENDER2 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.832 0.484 0.612 strong_typed_mention_match 0.761 0.443 0.560 strong_typed_all_match 0.750 0.414 0.533 strong_typed_link_match 0.792 0.545 0.646 strong_typed_nil_match 0.706 0.561 0.625 entity_match 0.804 0.468 0.592 mention_ceaf 0.771 0.449 0.567 typed_mention_ceaf 0.737 0.429 0.542 typed_mention_ceaf_plus 0.807 0.331 0.469 b_cubed 0.730 0.310 0.435 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.935 0.574 0.711 strong_mention_match 0.884 0.542 0.672 strong_typed_mention_match 0.809 0.496 0.615 strong_typed_all_match 0.780 0.487 0.599 strong_typed_link_match 0.919 0.530 0.672 strong_typed_nil_match 0.754 0.647 0.697 entity_match 0.871 0.534 0.662 mention_ceaf 0.829 0.508 0.630 typed_mention_ceaf 0.787 0.483 0.598 typed_mention_ceaf_plus 0.882 0.404 0.554 b_cubed 0.801 0.378 0.514 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.855 0.454 0.593 strong_mention_match 0.820 0.435 0.568 strong_typed_mention_match 0.760 0.404 0.527 strong_typed_all_match 0.747 0.383 0.506 strong_typed_link_match 0.808 0.487 0.608 strong_typed_nil_match 0.683 0.499 0.577 entity_match 0.821 0.436 0.569 mention_ceaf 0.792 0.420 0.549 typed_mention_ceaf 0.750 0.398 0.520 typed_mention_ceaf_plus 0.787 0.313 0.448 b_cubed 0.724 0.291 0.415 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.824 0.500 0.623 strong_mention_match 0.774 0.470 0.585 strong_typed_mention_match 0.698 0.423 0.527 strong_typed_all_match 0.702 0.355 0.471 strong_typed_link_match 0.691 0.617 0.652 strong_typed_nil_match 0.665 0.521 0.584 entity_match 0.725 0.440 0.548 mention_ceaf 0.694 0.421 0.524 typed_mention_ceaf 0.655 0.397 0.494 typed_mention_ceaf_plus 0.750 0.326 0.455 b_cubed 0.655 0.299 0.411 b_cubed_plus ************************************************************* Run ID: RPI_BLENDER3 Did the run access the live Web during the evaluation window: No Did the run use the Wikipedia links in the Reference KB: Yes Did the run use the relation/slot information encoded in the Reference KB: Yes Did the run use the text descriptions in the nodes of the Reference KB: No ALL English, Chinese, and Spanish documents: Prec Recall F1 Metric 0.833 0.475 0.605 strong_typed_mention_match 0.762 0.435 0.554 strong_typed_all_match 0.751 0.407 0.527 strong_typed_link_match 0.795 0.533 0.638 strong_typed_nil_match 0.707 0.550 0.619 entity_match 0.806 0.460 0.585 mention_ceaf 0.772 0.440 0.561 typed_mention_ceaf 0.739 0.421 0.537 typed_mention_ceaf_plus 0.809 0.321 0.460 b_cubed 0.732 0.300 0.426 b_cubed_plus ONLY English documents: Prec Recall F1 Metric 0.935 0.561 0.701 strong_mention_match 0.884 0.530 0.663 strong_typed_mention_match 0.811 0.486 0.608 strong_typed_all_match 0.782 0.475 0.591 strong_typed_link_match 0.921 0.527 0.670 strong_typed_nil_match 0.757 0.634 0.690 entity_match 0.871 0.522 0.653 mention_ceaf 0.829 0.497 0.621 typed_mention_ceaf 0.789 0.473 0.591 typed_mention_ceaf_plus 0.881 0.391 0.542 b_cubed 0.803 0.367 0.504 b_cubed_plus ONLY Chinese documents: Prec Recall F1 Metric 0.855 0.454 0.593 strong_mention_match 0.820 0.435 0.568 strong_typed_mention_match 0.760 0.404 0.527 strong_typed_all_match 0.747 0.383 0.506 strong_typed_link_match 0.808 0.487 0.608 strong_typed_nil_match 0.683 0.499 0.577 entity_match 0.821 0.436 0.569 mention_ceaf 0.792 0.420 0.549 typed_mention_ceaf 0.750 0.398 0.520 typed_mention_ceaf_plus 0.787 0.313 0.448 b_cubed 0.724 0.291 0.415 b_cubed_plus ONLY Spanish documents: Prec Recall F1 Metric 0.827 0.481 0.608 strong_mention_match 0.777 0.452 0.571 strong_typed_mention_match 0.698 0.406 0.514 strong_typed_all_match 0.703 0.343 0.461 strong_typed_link_match 0.691 0.584 0.633 strong_typed_nil_match 0.664 0.502 0.572 entity_match 0.729 0.424 0.536 mention_ceaf 0.698 0.406 0.513 typed_mention_ceaf 0.657 0.382 0.484 typed_mention_ceaf_plus 0.756 0.309 0.439 b_cubed 0.658 0.282 0.395 b_cubed_plus