Data

Queries

Participants will be provided with a set of 50 queries, together with the Freebase ID for each entity in these queries.

Documents

We will be using the Clueweb12 document collection. Participants can also use Clueweb12 Category B. However, all the runs will be evaluated using the same evaluation setup.



Sample Data release

While complete topic descriptions are being built, we are releasing three sample queries with indepth tasks and subtasks. These include queries, freebase entities and respective mids, task descriptions, a non-exhaustive list of subtasks. We have also included examples of possible participants submissions for the "Task Understanding" task that would be considered relevant for these subtasks. These topics can be found here.

Important notice: These examples are provided for a better understanding of the track structure and goals. For the test queries, participants will only receive queries and Freebase mids of entities in the queries. The task descriptions provided in the sample topics are not yet exhaustive. A more exhaustive list of subtasks will be constructed after pooling participants submissions and extracting subtasks towards the completion of each task (as expressed in the guidelines).