Title: Measuring the quality of search result summaries
Speaker: Rajan Patel
Affiliation: Google
Abstract: Measuring search result summaries is challenging. There are multiple dimensions of quality to balance, and summaries' effects on user behavior is often dominated by higher-order effects such as result ranking. In this talk, we'll discuss techniques for measuring result summaries. We'll walk through several different approaches we've tried at Google, using both human evaluation and click-based metrics. Beyond pros and cons for each approach, we'll identify some general considerations for evaluating result summaries.