TY  - JOUR
AU  - Joachims, Thorsten
AU  - Granka, Laura
AU  - Pan, Bing
AU  - Hembrooke, Helene
AU  - Radlinski, Filip
AU  - Gay, Geri
T1  - Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search
JO  - ACM Trans. Inf. Syst.
PY  - 2007/
VL  - 25
IS  - 2
SP  - 
EP  - 
UR  - http://portal.acm.org/citation.cfm?id=1229181
M3  - 10.1145/1229179.1229181
KW  - learning-to-rank
KW  - feedback
KW  - click
KW  - ranking
KW  - search
KW  - web
L1  - 
SN  - 
N1  - 
N1  - 
AB  - This article examines the reliability of implicit feedback generated from clickthrough data and query reformulations in World Wide Web (WWW) search. Analyzing the users' decision process using eyetracking and comparing implicit feedback against manual relevance judgments, we conclude that clicks are informative but biased. While this makes the interpretation of clicks as absolute relevance judgments difficult, we show that relative preferences derived from clicks are reasonably accurate on average. We find that such relative preferences are accurate not only between results from an individual query, but across multiple sets of results within chains of query reformulations.
ER  -

TY  - CONF
AU  - Joachims, Thorsten
AU  - Granka, Laura
AU  - Pan, Bing
AU  - Hembrooke, Helene
AU  - Gay, Geri
A2  - 
T1  - Accurately interpreting clickthrough data as implicit feedback
T2  - Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
PB  - ACM
CY  - New York, NY, USA
PY  - 2005/
M2  - 
VL  - 
IS  - 
SP  - 154
EP  - 161
UR  - http://doi.acm.org/10.1145/1076034.1076063
M3  - 10.1145/1076034.1076063
KW  - implicit
KW  - feedback
KW  - click
KW  - study
KW  - search
KW  - data
KW  - evaluation
KW  - user
KW  - web
L1  - 
SN  - 1-59593-034-5
N1  - 
N1  - 
AB  - This paper examines the reliability of implicit feedback generated from clickthrough data in WWW search. Analyzing the users' decision process using eyetracking and comparing implicit feedback against manual relevance judgments, we conclude that clicks are informative but biased. While this makes the interpretation of clicks as absolute relevance judgments difficult, we show that relative preferences derived from clicks are reasonably accurate on average.
ER  -

TY  - CONF
AU  - Joachims, Thorsten
AU  - Granka, Laura
AU  - Pan, Bing
AU  - Hembrooke, Helene
AU  - Gay, Geri
A2  - 
T1  - Accurately interpreting clickthrough data as implicit feedback
T2  - SIGIR '05: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
PB  - ACM
CY  - New York, NY, USA
PY  - 2005/
M2  - 
VL  - 
IS  - 
SP  - 154
EP  - 161
UR  - http://portal.acm.org/citation.cfm?id=1076063
M3  - 10.1145/1076034.1076063
KW  - learning-to-rank
KW  - feedback
KW  - click
KW  - ranking
KW  - search
KW  - web
L1  - 
SN  - 1-59593-034-5
N1  - 
N1  - 
AB  - This paper examines the reliability of implicit feedback generated from clickthrough data in WWW search. Analyzing the users' decision process using eyetracking and comparing implicit feedback against manual relevance judgments, we conclude that clicks are informative but biased. While this makes the interpretation of clicks as absolute relevance judgments difficult, we show that relative preferences derived from clicks are reasonably accurate on average.
ER  -