TAC 2008 Opinion QA Task
Opinion QA Task Guidelines
Opinion QA Test Data:
Questions
(XML file)
Blog06 document collection
(obtain from
University of Glasgow
)
Ranking of top 1000 documents per target
(optional; gzipped file)
check_qa.pl
: Perl script for checking QA submission file
Opinion QA Evaluation Results:
README.QA.2008
rigid_qrels
: Rigid list judgments for all responses
table.rigidlists
: Rigid list summary table
QA2008_nuggets.pyramid
: Squishy list nugget pyramid
squishy_qrels
: Squishy list judgments for all responses
table.squishylists.pyramid
: Squishy list summary table
table.series
: Per-Series combined score summary table
Opinion QA runs submitted:
README.QA.runs.2008
QA runs and individual evaluation results
(gzipped tar file)
TAC 2008 Opinion Summarization Pilot
Opinion Summarization Pilot Guidelines
Opinion Summarization Test Data:
Test targets/questions/documents
(XML file)
Test document sets
(gzipped tar file)
Optional
answer-snippets
Opinion Summarization Evaluation Results:
README.OpSumm08
OpSumm08_summaries.tar.gz
: Evaluated summaries (gzipped tar file)
OpSumm08_nuggets.pyramid
: Nugget pyramid
OpSumm08_nugget_judgments
: Nuggets matched for each summary
OpSumm08_pyramid_scores
: Nugget pyramid scores for each summary
OpSumm08_quality_scores
: Fluency/readability and overall responsiveness for each summary
OpSumm08.avg_scores
: Average scores for each run
Opinion Summarization Participants
-- txt file mapping between numeric summarizer ID and run name (team ID + priority)
Last updated:
Comments to:
[email protected]