Autoeval: An evaluation methodology for evaluating query suggestions using query logs

M. Dyaa Albakour; Udo Kruschwitz; Nikolaos Nanas; Yunhyong Kim; Dawei Song; Maria Fasli; Anne De Roeck

doi:10.1007/978-3-642-20161-5_60

Autoeval: An evaluation methodology for evaluating query suggestions using query logs

M. Dyaa Albakour, Udo Kruschwitz, Nikolaos Nanas, Yunhyong Kim, Dawei Song, Maria Fasli, Anne De Roeck

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

10 引用（Scopus）

摘要

User evaluations of search engines are expensive and not easy to replicate. The problem is even more pronounced when assessing adaptive search systems, for example system-generated query modification suggestions that can be derived from past user interactions with a search engine. Automatically predicting the performance of different modification suggestion models before getting the users involved is therefore highly desirable. AutoEval is an evaluation methodology that assesses the quality of query modifications generated by a model using the query logs of past user interactions with the system. We present experimental results of applying this methodology to different adaptive algorithms which suggest that the predicted quality of different algorithms is in line with user assessments. This makes AutoEval a suitable evaluation framework for adaptive interactive search engines.

源语言	英语
主期刊名	Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings
编辑	Paul Clough, Colum Foley, Cathal Gurrin, Hyowon Lee, Gareth J.F. Jones, Wessel Kraaij, Vanessa Murdoch
出版商	Springer Verlag
页	605-610
页数	6
ISBN（印刷版）	9783642201608
DOI	https://doi.org/10.1007/978-3-642-20161-5_60
出版状态	已出版 - 2011
已对外发布	是
活动	33rd European Conference on Information Retrieval, ECIR 2011 - Dublin, 爱尔兰期限: 18 4月 2011 → 21 4月 2011

出版系列

姓名	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
卷	6611 LNCS
ISSN（印刷版）	0302-9743
ISSN（电子版）	1611-3349

会议

会议	33rd European Conference on Information Retrieval, ECIR 2011
国家/地区	爱尔兰
市	Dublin
时期	18/04/11 → 21/04/11

访问文件

10.1007/978-3-642-20161-5_60

其它文件与链接

链接到 Scopus 的出版物

引用此

Albakour, M. D., Kruschwitz, U., Nanas, N., Kim, Y., Song, D., Fasli, M., & De Roeck, A. (2011). Autoeval: An evaluation methodology for evaluating query suggestions using query logs. 在 P. Clough, C. Foley, C. Gurrin, H. Lee, G. J. F. Jones, W. Kraaij, & V. Murdoch (编辑), Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings (页码 605-610). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); 卷 6611 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-642-20161-5_60

Albakour, M. Dyaa ; Kruschwitz, Udo ; Nanas, Nikolaos 等. / Autoeval : An evaluation methodology for evaluating query suggestions using query logs. Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings. 编辑 / Paul Clough ; Colum Foley ; Cathal Gurrin ; Hyowon Lee ; Gareth J.F. Jones ; Wessel Kraaij ; Vanessa Murdoch. Springer Verlag, 2011. 页码 605-610 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{a9eb8a282f2343dfaa15185e437749a4,

title = "Autoeval: An evaluation methodology for evaluating query suggestions using query logs",

abstract = "User evaluations of search engines are expensive and not easy to replicate. The problem is even more pronounced when assessing adaptive search systems, for example system-generated query modification suggestions that can be derived from past user interactions with a search engine. Automatically predicting the performance of different modification suggestion models before getting the users involved is therefore highly desirable. AutoEval is an evaluation methodology that assesses the quality of query modifications generated by a model using the query logs of past user interactions with the system. We present experimental results of applying this methodology to different adaptive algorithms which suggest that the predicted quality of different algorithms is in line with user assessments. This makes AutoEval a suitable evaluation framework for adaptive interactive search engines.",

author = "Albakour, {M. Dyaa} and Udo Kruschwitz and Nikolaos Nanas and Yunhyong Kim and Dawei Song and Maria Fasli and {De Roeck}, Anne",

note = "Publisher Copyright: {\textcopyright} Springer-Verlag Berlin Heidelberg 2011.; 33rd European Conference on Information Retrieval, ECIR 2011 ; Conference date: 18-04-2011 Through 21-04-2011",

year = "2011",

doi = "10.1007/978-3-642-20161-5_60",

language = "English",

isbn = "9783642201608",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "605--610",

editor = "Paul Clough and Colum Foley and Cathal Gurrin and Hyowon Lee and Jones, {Gareth J.F.} and Wessel Kraaij and Vanessa Murdoch",

booktitle = "Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings",

address = "Germany",

}

Albakour, MD, Kruschwitz, U, Nanas, N, Kim, Y, Song, D, Fasli, M & De Roeck, A 2011, Autoeval: An evaluation methodology for evaluating query suggestions using query logs. 在 P Clough, C Foley, C Gurrin, H Lee, GJF Jones, W Kraaij & V Murdoch (编辑), Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 卷 6611 LNCS, Springer Verlag, 页码 605-610, 33rd European Conference on Information Retrieval, ECIR 2011, Dublin, 爱尔兰, 18/04/11. https://doi.org/10.1007/978-3-642-20161-5_60

Autoeval: An evaluation methodology for evaluating query suggestions using query logs. / Albakour, M. Dyaa; Kruschwitz, Udo; Nanas, Nikolaos 等.
Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings. 编辑 / Paul Clough; Colum Foley; Cathal Gurrin; Hyowon Lee; Gareth J.F. Jones; Wessel Kraaij; Vanessa Murdoch. Springer Verlag, 2011. 页码 605-610 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); 卷 6611 LNCS).

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

TY - GEN

T1 - Autoeval

T2 - 33rd European Conference on Information Retrieval, ECIR 2011

AU - Albakour, M. Dyaa

AU - Kruschwitz, Udo

AU - Nanas, Nikolaos

AU - Kim, Yunhyong

AU - Song, Dawei

AU - Fasli, Maria

AU - De Roeck, Anne

N1 - Publisher Copyright: © Springer-Verlag Berlin Heidelberg 2011.

PY - 2011

Y1 - 2011

N2 - User evaluations of search engines are expensive and not easy to replicate. The problem is even more pronounced when assessing adaptive search systems, for example system-generated query modification suggestions that can be derived from past user interactions with a search engine. Automatically predicting the performance of different modification suggestion models before getting the users involved is therefore highly desirable. AutoEval is an evaluation methodology that assesses the quality of query modifications generated by a model using the query logs of past user interactions with the system. We present experimental results of applying this methodology to different adaptive algorithms which suggest that the predicted quality of different algorithms is in line with user assessments. This makes AutoEval a suitable evaluation framework for adaptive interactive search engines.

AB - User evaluations of search engines are expensive and not easy to replicate. The problem is even more pronounced when assessing adaptive search systems, for example system-generated query modification suggestions that can be derived from past user interactions with a search engine. Automatically predicting the performance of different modification suggestion models before getting the users involved is therefore highly desirable. AutoEval is an evaluation methodology that assesses the quality of query modifications generated by a model using the query logs of past user interactions with the system. We present experimental results of applying this methodology to different adaptive algorithms which suggest that the predicted quality of different algorithms is in line with user assessments. This makes AutoEval a suitable evaluation framework for adaptive interactive search engines.

UR - http://www.scopus.com/inward/record.url?scp=84973247809&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-20161-5_60

DO - 10.1007/978-3-642-20161-5_60

M3 - Conference contribution

AN - SCOPUS:84973247809

SN - 9783642201608

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 605

EP - 610

BT - Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings

A2 - Clough, Paul

A2 - Foley, Colum

A2 - Gurrin, Cathal

A2 - Lee, Hyowon

A2 - Jones, Gareth J.F.

A2 - Kraaij, Wessel

A2 - Murdoch, Vanessa

PB - Springer Verlag

Y2 - 18 April 2011 through 21 April 2011

ER -

Albakour MD, Kruschwitz U, Nanas N, Kim Y, Song D, Fasli M 等. Autoeval: An evaluation methodology for evaluating query suggestions using query logs. 在 Clough P, Foley C, Gurrin C, Lee H, Jones GJF, Kraaij W, Murdoch V, 编辑, Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Proceedings. Springer Verlag. 2011. 页码 605-610. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-20161-5_60

Autoeval: An evaluation methodology for evaluating query suggestions using query logs

摘要

出版系列

会议

访问文件

其它文件与链接

指纹

引用此