Abstract
In evaluation of ranked retrieval, the usual assumption is that the documents to be searched can be indexed before the query is received and the search is performed. The NTCIR-18 SUSHI Pilot Task, by contrast, models the case in which only a small sample of the documents to be searched can be indexed before the query is received. This task model arises in the context of searching within large archives of paper documents, for example. The stark difference in what can be indexed before the query is received has consequences for both task design and evaluation design, both of which are discussed in this paper.
| Original language | English |
|---|---|
| Journal | CEUR Workshop Proceedings |
| Volume | 3854 |
| Publication status | Published - 2024 |
| Event | Joint of the 1st Workshop on Evaluation Methodologies, Testbeds and Community for Information Access Research and the 1st Workshop on User Modelling in Conversational Information Retrieval, EMTCIR_UM-CIR 2024 - Tokyo, Japan Duration: Dec 12 2024 → … |
All Science Journal Classification (ASJC) codes
- General Computer Science