NTCIR-3 WEB: An evaluation workshop for web retrieval

Koji Eguchi, Keizo Oyama, Emi Ishida, Noriko Kando, Kazuko Kuriyama

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)


The authors conducted the Web Retrieval Task ('NTCIR-3 WEB') from 2001 to 2002 at the Third NTCIR Workshop. In the NTCIR-3 WEB, they attempted to assess the retrieval effectiveness of Web search engine systems using a common data set, and to build re-usable test collections that are suitable for evaluating Web information retrieval systems. With these objectives, they evaluated on searches using various types of user input, user models and document models. As the document data sets, they constructed 100-gigabyte and 10-gigabyte document collections that were gathered from the '.jp' domain. The user input was given as query term(s), sentence, and document(s). They assumed two user models where comprehensive relevant documents are required, and where precision of the top-ranked results is emphasized. They also assumed several document models, such as a document as an individual page, and a document as a page set connected by hyperlinks. This paper describes an overview of the test collections constructed in the NTCIR-3 WEB, the proposed evaluation methods, and the evaluation results. The evaluation results suggested that the link-based techniques can perform more effectively when short queries are input.

Original languageEnglish
Pages (from-to)31-56
Number of pages26
JournalNII Journal
Issue number6
Publication statusPublished - Mar 2003
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Library and Information Sciences


Dive into the research topics of 'NTCIR-3 WEB: An evaluation workshop for web retrieval'. Together they form a unique fingerprint.

Cite this