Tasks track lies at the intersection of Diversity and Session Track at TREC. The primary goals of the track are to evaluate system's understanding of tasks users aim to achieve and evaluate relevance of retrieved documents with respect to underlying tasks in query.
Research in Information Retrieval has traditionally focused on serving the best results for a single query, ignoring the reasons (or the task) that might have motivated the user to submit that query. Often times search engines are used to complete complex tasks (information needs); achieving these tasks with current search engines requires users to issue multiple queries. For example, booking travel to a location such as London could require the user to submit various queries such as flights to London, hotels in London, points of interest around London, etc. Similarly, a person who is trying to organize a wedding would need to issue separate queries in order to locate stores to buy a wedding gown, arrange catering, book honeymoon, etc. In some cases users may not even be aware of all the subtasks they need to achieve to satisfy their information need, which makes search an even more difficult experience.
Ideally, a search engine should be able to understand the reason that caused the user to submit a query (i.e., the actual task that caused the query to be issued), and rather than just showing results relevant to the query submitted, the search engine should be able to guide the user to achieve their task by incorporating the information about the actual information need. The goal of this track is to devise evaluation methodologies for evaluating the quality of task based information retrieval systems.

Mailing list: tasks-track@googlegroups.com

Subscribe at https://groups.google.com/forum/#!forum/tasks-track


Emine Yilmaz (e.yilmaz@cs.ucl.ac.uk)

Evangelos Kanoulas (e.kanoulas@uva.nl)

Nick Craswell (nickcr@microsoft.com)

Peter Bailey (pbailey@microsoft.com)

Ben Carterette (carteret@cis.udel.edu)

Rishabh Mehrotra (r.mehrotra@cs.ucl.ac.uk)