Tasks track lies at the intersection of Diversity and Session Track at TREC. The primary goals of the track are to evaluate system's understanding of tasks users aim to achieve and evaluate relevance of retrieved documents with respect to underlying tasks in query.
Research in Information Retrieval has traditionally focused on serving the best results for a single query, ignoring the reasons (or the task) that might have motivated the user to submit that query. Often times search engines are used to complete complex tasks (information needs); achieving these tasks with current search engines requires users to issue multiple queries. For example, booking travel to a location such as London could require the user to submit various queries such as flights to London, hotels in London, points of interest around London, etc. Similarly, a person who is trying to organize a wedding would need to issue separate queries in order to locate stores to buy a wedding gown, arrange catering, book honeymoon, etc. In some cases users may not even be aware of all the subtasks they need to achieve to satisfy their information need, which makes search an even more difficult experience.
Ideally, a search engine should be able to understand the reason that caused the user to submit a query (i.e., the actual task that caused the query to be issued), and rather than just showing results relevant to the query submitted, the search engine should be able to guide the user to achieve their task by incorporating the information about the actual information need. The goal of this track is to devise evaluation methodologies for evaluating the quality of task based information retrieval systems.
Sample Data Release
We are releasing 3 sample queries while a more extensive topic set is being developed. As per the guidelines, for test data, please note that only the query and freebase id will be released.
Sample topics can be found here.
Test Data Release
We have released the test queries on active participants page. If you have not expressed interest to participate in Tasks Track via Trec Website, please do so here.
Submission Deadline extended!
We are extending the deadline for run submissions! Now results can be submitted until 1st September 2015. Run format has been posted here.
Mailing list: firstname.lastname@example.org
Subscribe at https://groups.google.com/d/forum/tasktrack2015
Submit Task Topics at http://goo.gl/forms/QmmkTMXiod