Information for TAC 2009 Track Participants
TAC 2009 has three tracks:
The exact definition of the
tasks to be performed in each track for TAC 2009 is formulated and
discussed on the track mailing list. Registering to participate in a TAC
track does not automatically add you to the track's mailing
list; if you are not already on the mailing list, you must subscribe
yourself by following the instructions for contacting that mailing
list as given in the track web page. If you were previously
subscribed to the mailing list, you do not have to re-subscribe; track
mailing lists are for anyone interested in the track, rather than
specifically for TAC participants, and thus carry over from year to
year. For questions about the track, send mail to the track
coordinator (or post the question to the track mailing list once you
join).
Each track has its own set of deadlines for submitting results,
which will be given in the final track guidelines. Most results are due
by very early August to give assessors time to judge the results.
Tracks and tasks with smaller assessing constraints (such as the RTE
Track) may have later deadlines.
For most tracks, the
judgments against which each system's output will be scored will be
made by experienced assessors based on manual analysis and the output
of all track participants.
Track participants must abide by the Policies and
Guidelines for Track Participants. All results submitted to TAC
are archived on the TAC web site and all evaluations are published in
the conference proceedings. Dissemination of TAC work and results
other than in the conference proceedings is welcomed, but the
conditions of participation specifically preclude any advertising
claims based on TAC results.
Data and User Agreements
Whenever possible, data that are required for the tracks are
distributed through the track web pages; data are mailed as
physical disks only if they cannot be made available for download.
Test materials for most tracks are distributed by NIST
at no cost to track participants. However, some tracks may use
special collections that are available from other organizations for a
fee; the individual track guidelines will specify instructions for
obtaining such data.
Most data is kept in password-protected areas of the TAC web site.
Password-protected areas include the current year's test data and
evaluation results, and any data that is copyrighted. Access to TAC
2009 data is restricted to registered TAC 2009 participants who have
submitted all the required User Agreement forms. Each
password-protected area of the web site will indicate which signed
User Agreements are required for access. Each participating team will
provide a Team ID and receive a Team Password upon registration. Once
the required User Agreement forms have been received for that team,
NIST will activate the Team ID and Password to give access to the
corresponding password-protected web pages. Teams that have
participated in past TAC cycles must register and obtain a new
password for 2009. Teams must also submit new (current) User
Agreement forms for 2009.
Track participants should follow the instructions given in the User Agreements page for
submitting the "Agreement Concerning Dissemination of TAC Results" and
any additional User Agreements that are required for the particular
tracks of interest. When submitting forms, in addition to the
information requested in the User Agreements page,
please include the email address of the main TAC 2009 contact
person for the team, and a comment saying that the form is from a TAC 2009
registered participant. NIST needs a signed physical copy of the
agreements before the team can be given any data.
Much of the past data from TREC, DUC and TAC are available from NIST and
may be useful for system development for some of the TAC tracks.
Beginning March 3, registered TAC participants may also request disks
containing entire document collections that might be useful (but not
required) for some of the tracks. These
document disks consist of the TIPSTER set
(disks 1-3) and the TREC set (disks 4-5). Participants should follow the
instructions in the User
Agreements page for submitting the "Agreement Concerning
Dissemination of TAC Results" and any additional User Agreements that
are required for the particular disks that are being requested. NIST
will ship the CD-ROMs at no cost to participants who have
returned all required forms. No disks will be shipped before March
3.
Schedule
TAC 2009 Schedule |
Beginning March 3 | Submit signed User Agreements to NIST |
May 31 | Deadline for registration for track participation |
July - early September | Deadlines for results submission for most tracks |
September 25 | Deadline for workshop presentation proposals |
September 25 | Deadline for proposals for TAC 2010 tracks/tasks |
By early October | Release of individual evaluated results to participants |
October 7 | Notification of acceptance of presentation proposals |
October 22 | Deadline for participants' notebook papers, copyright release forms |
November 16-17 | TAC 2009 workshop in Gaithersburg, Maryland, USA |
February 25, 2010 | Extended Deadline for participants' final proceedings papers |