PUMA publications for /author/Ron%20Kohavihttps://puma.uni-kassel.de/author/Ron%20KohaviPUMA RSS feed for /author/Ron%20Kohavi2024-03-29T09:33:57+01:00Online Controlled Experiments: Introduction, Learnings, and Humbling Statisticshttps://puma.uni-kassel.de/bibtex/2aa31e13651d5d1eab42e449e55a0e745/jaeschkejaeschke2012-09-20T09:59:05+02:00recommender experiment statistics bing amazon 2012 keynote online industry recsys evaluation <span class="authorEditorList"><span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Ron Kohavi" itemprop="url" href="/author/Ron%20Kohavi"><span itemprop="name">R. Kohavi</span></a></span>. </span>(<em><span>12.09.2012<meta content="12.09.2012" itemprop="datePublished"/></span></em>)Thu Sep 20 09:59:05 CEST 2012sepOnline Controlled Experiments: Introduction, Learnings, and Humbling StatisticsIndustry keynote at ACM Recommender Systems2012recommender experiment statistics bing amazon 2012 keynote online industry recsys evaluation 12The web provides an unprecedented opportunity to accelerate innovation by evaluating ideas quickly and accurately using controlled experiments (e.g., A/B tests and their generalizations). Whether for front-end user-interface changes, or backend recommendation systems and relevance algorithms, online controlled experiments are now utilized to make data-driven decisions at Amazon, Microsoft, eBay, Facebook, Google, Yahoo, Zynga, and at many other companies. While the theory of a controlled experiment is simple, and dates back to Sir Ronald A. Fisher’s experiments at the Rothamsted Agricultural Experimental Station in England in the 1920s, the deployment and mining of online controlled experiments at scale—thousands of experiments now—has taught us many lessons. We provide an introduction, share real examples, key learnings, cultural challenges, and humbling statistics. A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selectionhttps://puma.uni-kassel.de/bibtex/27389c34380588234cb69bd5dfdb6bd2f/hothohotho2007-07-31T14:51:53+02:00cross learning validation <span class="authorEditorList"><span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Ron Kohavi" itemprop="url" href="/author/Ron%20Kohavi"><span itemprop="name">R. Kohavi</span></a></span>. </span><span itemtype="http://schema.org/Book" itemscope="itemscope" itemprop="isPartOf"><em><span itemprop="name">Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence</span>, </em></span><em>Seite <span itemprop="pagination">1137-1145</span>. </em><em><span itemprop="publisher">San Mateo, CA: Morgan Kaufmann</span>, </em>(<em><span>1995<meta content="1995" itemprop="datePublished"/></span></em>)Tue Jul 31 14:51:53 CEST 2007Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence1137-1145A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection1995cross learning validation WSD