TAC 2008 Opinion QA Task

• Opinion QA Task Guidelines
• Opinion QA Test Data:
• Questions (XML file)
• Blog06 document collection (obtain from University of Glasgow)
• Ranking of top 1000 documents per target (optional; gzipped file)
• check_qa.pl: Perl script for checking QA submission file
• Opinion QA Evaluation Results:
• README.QA.2008
• rigid_qrels: Rigid list judgments for all responses
• table.rigidlists: Rigid list summary table
• QA2008_nuggets.pyramid: Squishy list nugget pyramid
• squishy_qrels: Squishy list judgments for all responses
• table.squishylists.pyramid: Squishy list summary table
• table.series: Per-Series combined score summary table
• Opinion QA runs submitted:
• README.QA.runs.2008
• QA runs and individual evaluation results (gzipped tar file)

TAC 2008 Opinion Summarization Pilot

• Opinion Summarization Pilot Guidelines
• Opinion Summarization Test Data:
• Test targets/questions/documents (XML file)
• Test document sets (gzipped tar file)
• Optional answer-snippets
• Opinion Summarization Evaluation Results:
• README.OpSumm08
• OpSumm08_summaries.tar.gz: Evaluated summaries (gzipped tar file)
• OpSumm08_nuggets.pyramid: Nugget pyramid
• OpSumm08_nugget_judgments: Nuggets matched for each summary
• OpSumm08_pyramid_scores: Nugget pyramid scores for each summary
• OpSumm08_quality_scores: Fluency/readability and overall responsiveness for each summary
• OpSumm08.avg_scores: Average scores for each run
• Opinion Summarization Participants -- txt file mapping between numeric summarizer ID and run name (team ID + priority)