Atzmueller, M.; Becker, M.; Kibanov, M.; Scholz, C.; Doerfel, S.; Hotho, A.; Macek, B.-E.; Mitzlaff, F.; Mueller, J. & Stumme, G.: Ubicon and its applications for ubiquitous social computing. In: New Review of Hypermedia and Multimedia 20 (2014), Nr. 1, S. 53-77
[Volltext]
The combination of ubiquitous and social computing is an emerging
esearch area which integrates different but complementary methods,
echniques and tools. In this paper, we focus on the Ubicon platform,
ts applications, and a large spectrum of analysis results.

bicon provides an extensible framework for building and hosting applications
argeting both ubiquitous and social environments. We summarize the
rchitecture and exemplify its implementation using four real-world
pplications built on top of Ubicon. In addition, we discuss several
cientific experiments in the context of these applications in order
o give a better picture of the potential of the framework, and discuss
nalysis results using several real-world data sets collected utilizing
bicon.

Beel, J.; Langer, S.; Genzmehr, M.; Gipp, B.; Breitinger, C. & Nürnberger, A.: Research Paper Recommender System Evaluation: A Quantitative Literature Survey. , 2013

Adomavicius, G. & Zhang, J.: Impact of Data Characteristics on Recommender Systems Performance. In: ACM Trans. Manage. Inf. Syst. 3 (2012), Nr. 1, S. 3:1-3:17
[Volltext]
This article investigates the impact of rating data characteristics on the performance of several popular recommendation algorithms, including user-based and item-based collaborative filtering, as well as matrix factorization. We focus on three groups of data characteristics: rating space, rating frequency distribution, and rating value distribution. A sampling procedure was employed to obtain different rating data subsamples with varying characteristics; recommendation algorithms were used to estimate the predictive accuracy for each sample; and linear regression-based models were used to uncover the relationships between data characteristics and recommendation accuracy. Experimental results on multiple rating datasets show the consistent and significant effects of several data characteristics on recommendation accuracy.

Gemmell, J.; Schimoler, T.; Mobasher, B. & Burke, R.: Resource recommendation in social annotation systems: A linear-weighted hybrid approach. In: Journal of Computer and System Sciences 78 (2012), Nr. 4, S. 1160 - 1174
[Volltext]
Social annotation systems enable the organization of online resources with user-defined keywords. Collectively these annotations provide a rich information space in which users can discover resources, organize and share their finds, and connect to other users with similar interests. However, the size and complexity of these systems can lead to information overload and reduced utility for users. For these reasons, researchers have sought to apply the techniques of recommender systems to deliver personalized views of social annotation systems. To date, most efforts have concentrated on the problem of tag recommendation – personalized suggestions for possible annotations. Resource recommendation has not received the same systematic evaluation, in part because the task is inherently more complex. In this article, we provide a general formulation for the problem of resource recommendation in social annotation systems that captures these variants, and we evaluate two cases: basic resource recommendation and tag-specific resource recommendation. We also propose a linear-weighted hybrid framework for resource recommendation. Using six real-world datasets, we show that its integrative approach is essential for this recommendation task and provides the most adaptability given the varying data characteristics in different social annotation systems. We find that our algorithm is more effective than other more mathematically-complex techniques and has the additional advantages of flexibility and extensibility.

Konstan, J. & Riedl, J.: Recommender systems: from algorithms to user experience. In: User Modeling and User-Adapted Interaction 22 (2012), Nr. 1-2, S. 101-123
[Volltext]
Since their introduction in the early 1990’s, automated recommender systems have revolutionized the marketing and delivery of commerce and content by providing personalized recommendations and predictions over a variety of large and complex product offerings. In this article, we review the key advances in collaborative filtering recommender systems, focusing on the evolution from research concentrated purely on algorithms to research concentrated on the rich set of questions around the user experience with the recommender. We show through examples that the embedding of the algorithm in the user experience dramatically affects the value to the user of the recommender. We argue that evaluating the user experience of a recommender requires a broader set of measures than have been commonly used, and suggest additional measures that have proven effective. Based on our analysis of the state of the field, we identify the most important open research problems, and outline key challenges slowing the advance of the state of the art, and in some cases limiting the relevance of research to real-world applications.

leaong, S.: A survey of recommender systems for scientific papers. , 2012

Owen, S.: Mahout in action. Shelter Island, N.Y.: Manning Publications Co., 2012
[Volltext]

Heck, T.; Peters, I. & Stock, W. G.: Testing collaborative filtering against co-citation analysis and bibliographic coupling for academic author recommendation. Workshop on Recommender Systems and the Social Web (ACM RecSys'11). 2011

Jannach, D.: Recommender systems : an introduction. New York: Cambridge University Press, 2011
[Volltext]

Kim, H.-N. & El Saddik, A.: Personalized PageRank vectors for tag recommendations: inside FolkRank. Proceedings of the fifth ACM conference on Recommender systems. New York, NY, USA: ACM, 2011RecSys '11 , S. 45-52
[Volltext]
This paper looks inside FolkRank, one of the well-known folksonomy-based algorithms, to present its fundamental properties and promising possibilities for improving performance in tag recommendations. Moreover, we introduce a new way to compute a differential approach in FolkRank by representing it as a linear combination of the personalized PageRank vectors. By the linear combination, we present FolkRank's probabilistic interpretation that grasps how FolkRank works on a folksonomy graph in terms of the random surfer model. We also propose new FolkRank-like methods for tag recommendations to efficiently compute tags' rankings and thus reduce expensive computational cost of FolkRank. We show that the FolkRank approaches are feasible to recommend tags in real-time scenarios as well. The experimental evaluations show that the proposed methods provide fast tag recommendations with reasonable quality, as compared to FolkRank. Additionally, we discuss the diversity of the top n tags recommended by FolkRank and its variants.

Kubatz, M.; Gedikli, F. & Jannach, D.: LocalRank - Neighborhood-Based, Fast Computation of Tag Recommendations. In: Huemer, C. & Setzer, T. (Hrsg.): E-Commerce and Web Technologies. Springer Berlin Heidelberg, 2011 (Lecture Notes in Business Information Processing 85), S. 258-269
[Volltext]
On many modern Web platforms users can annotate the available online resources with freely-chosen tags. This Social Tagging data can then be used for information organization or retrieval purposes. Tag recommenders in that context are designed to help the online user in the tagging process and suggest appropriate tags for resources with the purpose to increase the tagging quality. In recent years, different algorithms have been proposed to generate tag recommendations given the ternary relationships between users, resources, and tags. Many of these algorithms however suffer from scalability and performance problems, including the popular

Montañés, E.; Ramón Quevedo, J.; Díaz, I.; Cortina, R.; Alonso, P. & Ranilla, J.: TagRanker: learning to recommend ranked tags. In: Logic Journal of IGPL 19 (2011), Nr. 2, S. 395-404
[Volltext]
In a social network, recommenders are highly demanded since they provide user interests in order to construct user profiles. This user profiles might be valuable to be exploited in business management or marketing, for instance. Basically, a tag recommender provides to users a set keywords that describe certain resources. The existing approaches require exploiting content information or they just provide a set of tags without any kind of preference order. This article proposes TagRanker, a tag recommender based on logistic regression that is free of exploiting content information. In addition, it gives a ranking of certain tags and learns just from the relations among users, resources and tags previously posted avoiding the cost of exploiting the content of the resources. An adequate evaluation measure for this specific kind of ranking is also proposed, since the existing ones just consider the tags as coming from a classification. The experiments on several data sets show that TagRanker can effectively recommend relevant tags outperforming the performance of a benchmark of Tag Recommender Systems.

Pariser, E.: The filter bubble : what the Internet is hiding from you. New York: Penguin Press, 2011
[Volltext]
In December 2009, Google began customizing its search results for all users, and we entered a new era of personalization. With little notice or fanfare, our online experience is changing as the web sites we visit are increasingly tailoring themselves to us. In this engaging and visionary book, MoveOn.org board president Eli Pariser lays bare the personalization that is already taking place on every major web site, from Facebook to AOL to ABC News. As Pariser reveals, this new trend is nothing short of an invisible revolution in how we consume information, one that will shape how we learn, what we know, and even how our democracy works. The race to collect as much personal data about us as possible, and to tailor our online experience accordingly, is now the defining battle for today's internet giants like Google, Facebook, Apple, and Microsoft. Behind the scenes, a burgeoning industry of data companies is tracking our personal information-from our political leanings to the hiking boots we just browsed on Zappos-to sell to advertisers. As a result, we will increasingly each live in our own unique information universe-what Pariser calls "the filter bubble." We will receive mainly news that is pleasant and familiar and confirms our beliefs-and since these filters are invisible, we won't know what is being hidden from us. Out past interests will determine what we are exposed to in the future, leaving less room for the unexpected encounters that spark creativity, innovation, and the democratic exchange of ideas. Drawing on interviews with both cyberskeptics and cyberoptimists, from the cofounder of OkCupid, an algorithmically driven dating web site, to one of the chief visionaries of the U.S. information warfare, The Filter Bubble tells the story of how the internet, a medium built around the open flow of ideas, is closing in on itself under the pressure of commerce and "monetization." It peeks behind the curtain at the server farms, algorithms, and geeky entrepreneurs that have given us this new reality and investigates the consequences of corporate power in the digital age. The Filter Bubble reveals how personalization could undermine the internet's original purpose as an open platform for the spread of ideas and leave us all in an isolated, echoing world. But it is not too late to change course. Pariser lays out a new vision for the web, one that embraces the benefits of technology without turning a blind eye to its negative consequences and will ensure that the internet lives up to its transformative promise.

Shani, G. & Gunawardana, A.: Evaluating recommendation systems. In: Recommender Systems Handbook (2011), S. 257-297
[Volltext]

Cremonesi, P.; Koren, Y. & Turrin, R.: Performance of Recommender Algorithms on Top-n Recommendation Tasks. Proceedings of the Fourth ACM Conference on Recommender Systems. New York, NY, USA: ACM, 2010RecSys '10 , S. 39-46
[Volltext]
In many commercial systems, the 'best bet' recommendations are shown, but the predicted rating values are not. This is usually referred to as a top-N recommendation task, where the goal of the recommender system is to find a few specific items which are supposed to be most appealing to the user. Common methodologies based on error metrics (such as RMSE) are not a natural fit for evaluating the top-N recommendation task. Rather, top-N performance can be directly measured by alternative methodologies based on accuracy metrics (such as precision/recall). An extensive evaluation of several state-of-the art recommender algorithms suggests that algorithms optimized for minimizing RMSE do not necessarily perform as expected in terms of top-N recommendation task. Results show that improvements in RMSE often do not translate into accuracy improvements. In particular, a naive non-personalized algorithm can outperform some common recommendation approaches and almost match the accuracy of sophisticated algorithms. Another finding is that the very few top popular items can skew the top-N performance. The analysis points out that when evaluating a recommender algorithm on the top-N recommendation task, the test set should be chosen carefully in order to not bias accuracy metrics towards non-personalized solutions. Finally, we offer practitioners new variants of two collaborative filtering algorithms that, regardless of their RMSE, significantly outperform other recommender algorithms in pursuing the top-N recommendation task, with offering additional practical advantages. This comes at surprise given the simplicity of these two methods.

Gedikli, F. & Jannach, D.: Rating items by rating tags. In: Systems and the Social Web at ACM (2010),

Priem, J. & Hemminger, B. H.: Scientometrics 2.0: New metrics of scholarly impact on the social Web. , 2010
[Volltext]
The growing flood of scholarly literature is exposing the weaknesses of current, citation-based methods of evaluating and filtering articles. A novel and promising approach is to examine the use and citation of articles in a new forum: Web 2.0 services like social bookmarking and microblogging. Metrics based on this data could build a “Scientometics 2.0,” supporting richer and more timely pictures of articles' impact. This paper develops the most comprehensive list of these services to date, assessing the potential value and availability of data from each. We also suggest the next steps toward building and validating metrics drawn from the social Web.

Rendle, S. & Schmidt-Thieme, L.: Pairwise interaction tensor factorization for personalized tag recommendation. Proceedings of the third ACM international conference on Web search and data mining. New York, NY, USA: ACM, 2010WSDM '10 , S. 81-90
[Volltext]
Tagging plays an important role in many recent websites. Recommender systems can help to suggest a user the tags he might want to use for tagging a specific item. Factorization models based on the Tucker Decomposition (TD) model have been shown to provide high quality tag recommendations outperforming other approaches like PageRank, FolkRank, collaborative filtering, etc. The problem with TD models is the cubic core tensor resulting in a cubic runtime in the factorization dimension for prediction and learning.</p> <p>In this paper, we present the factorization model PITF (Pairwise Interaction Tensor Factorization) which is a special case of the TD model with linear runtime both for learning and prediction. PITF explicitly models the pairwise interactions between users, items and tags. The model is learned with an adaption of the Bayesian personalized ranking (BPR) criterion which originally has been introduced for item recommendation. Empirically, we show on real world datasets that this model outperforms TD largely in runtime and even can achieve better prediction quality. Besides our lab experiments, PITF has also won the ECML/PKDD Discovery Challenge 2009 for graph-based tag recommendation.

Bogers, T.: Recommender Systems for Social Bookmarking. Tilburg, The Netherlands, Tilburg University, 2009
[Volltext]
Recommender systems belong to a class of personalized information filtering technologies that aim to identify which items in a collection might be of interest to a particular user. Recommendations can be made using a variety of information sources related to both the user and the items: past user preferences, demographic information, item popularity, the metadata characteristics of the products, etc. Social bookmarking websites, with their emphasis on open collaborative information access, offer an ideal scenario for the application of recommender systems technology. They allow users to manage their favorite bookmarks online through a web interface and, in many cases, allow their users to tag the content they have added to the system with keywords. The underlying application then makes all information sharable among users. Examples of social bookmarking services include Delicious, Diigo, Furl, CiteULike, and BibSonomy. In my Ph.D. thesis I describe the work I have done on item recommendation for social bookmarking, i.e., recommending interesting bookmarks to users based on the content they bookmarked in the past. In my experiments I distinguish between two types of information sources. The first one is usage data contained in the folksonomy, which represents the past selections and transactions of all users, i.e., who added which items, and with what tags. The second information source is the metadata describing the bookmarks or articles on a social bookmarking website, such as title, description, authorship, tags, and temporal and publication-related metadata. I compare and combine the content-based aspect with the more common usage-based approaches. I evaluate my approaches on four data sets constructed from three different social bookmarking websites: BibSonomy, CiteULike, and Delicious. In addition, I investigate different combination methods for combining different algorithms and show which of those methods can successfully improve recommendation performance. Finally, I consider two growing pains that accompany the maturation of social bookmarking websites: spam and duplicate content. I examine how widespread each of these problems are for social bookmarking and how to develop effective automatic methods for detecting such unwanted content. Finally, I investigate the influence spam and duplicate content can have on item recommendation.

Jäschke, R.; Eisterlehner, F.; Hotho, A. & Stumme, G.: Testing and Evaluating Tag Recommenders in a Live System. RecSys '09: Proceedings of the 2009 ACM Conference on Recommender Systems. New York, NY, USA: ACM, 2009

The challenge to provide tag recommendations for collaborative tagging systems has attracted quite some attention of researchers lately. However, most research focused on the evaluation and development of appropriate methods rather than tackling the practical challenges of how to integrate recommendation methods into real tagging systems, record and evaluate their performance. In this paper we describe the tag recommendation framework we developed for our social bookmark and publication sharing system BibSonomy. With the intention to develop, test, and evaluate recommendation algorithms and supporting cooperation with researchers, we designed the framework to be easily extensible, open for a variety of methods, and usable independent from BibSonomy. Furthermore, this paper presents a �rst evaluation of two exemplarily deployed recommendation methods.

Landia, N. & Anand, S.: Personalised Tag Recommendation. In: Recommender Systems & the Social Web (2009),
[Volltext]

Rendle, S. & Schmidt-Thieme, L.: Factor Models for Tag Recommendation in BibSonomy. In: Eisterlehner, F.; Hotho, A. & Jäschke, R. (Hrsg.): ECML PKDD Discovery Challenge 2009 (DC09). Bled, Slovenia: CEUR Workshop Proceedings, 2009 (497), S. 235-242
[Volltext]
This paper describes our approach to the ECML/PKDD Discovery Challenge 2009. Our approach is a pure statistical model taking no content information into account. It tries to find latent interactions between users, items and tags by factorizing the observed tagging data. The factorization model is learned by the Bayesian Personal Ranking method (BPR) which is inspired by a Bayesian analysis of personalized ranking with missing data. To prevent overfitting, we ensemble the models over several iterations and hyperparameters. Finally, we enhance the top-n lists by estimating how many tags to recommend.

Hotho, A.; Jäschke, R.; Schmitz, C. & Stumme, G.: Trend detection in folksonomies. Proceedings of the First international conference on Semantic and Digital Media Technologies. Berlin, Heidelberg: Springer-Verlag, 2006SAMT'06 , S. 56-70
[Volltext]
As the number of resources on the web exceeds by far the number of documents one can track, it becomes increasingly difficult to remain up to date on ones own areas of interest. The problem becomes more severe with the increasing fraction of multimedia data, from which it is difficult to extract some conceptual description of their contents.</p> <p>One way to overcome this problem are social bookmark tools, which are rapidly emerging on the web. In such systems, users are setting up lightweight conceptual structures called folksonomies, and overcome thus the knowledge acquisition bottleneck. As more and more people participate in the effort, the use of a common vocabulary becomes more and more stable. We present an approach for discovering topic-specific trends within folksonomies. It is based on a differential adaptation of the PageRank algorithm to the triadic hypergraph structure of a folksonomy. The approach allows for any kind of data, as it does not rely on the internal structure of the documents. In particular, this allows to consider different data types in the same analysis step. We run experiments on a large-scale real-world snapshot of a social bookmarking system.

McNee, S. M.; Riedl, J. & Konstan, J. A.: Being accurate is not enough: how accuracy metrics have hurt recommender systems. CHI '06 extended abstracts on Human factors in computing systems. New York, NY, USA: ACM, 2006CHI EA '06 , S. 1097-1101
[Volltext]
Recommender systems have shown great potential to help users find interesting and relevant items from within a large information space. Most research up to this point has focused on improving the accuracy of recommender systems. We believe that not only has this narrow focus been misguided, but has even been detrimental to the field. The recommendations that are most accurate according to the standard metrics are sometimes not the recommendations that are most useful to users. In this paper, we propose informal arguments that the recommender community should move beyond the conventional accuracy metrics and their associated experimental methodologies. We propose new user-centric directions for evaluating recommender systems.

McNee, S. M.; Kapoor, N. & Konstan, J. A.: Don't look stupid: avoiding pitfalls when recommending research papers. Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work. New York, NY, USA: ACM, 2006CSCW '06 , S. 171-180
[Volltext]
If recommenders are to help people be more productive, they need to support a wide variety of real-world information seeking tasks, such as those found when seeking research papers in a digital library. There are many potential pitfalls, including not knowing what tasks to support, generating recommendations for the wrong task, or even failing to generate any meaningful recommendations whatsoever. We posit that different recommender algorithms are better suited to certain information seeking tasks. In this work, we perform a detailed user study with over 130 users to understand these differences between recommender algorithms through an online survey of paper recommendations from the ACM Digital Library. We found that pitfalls are hard to avoid. Two of our algorithms generated 'atypical' recommendations recommendations that were unrelated to their input baskets. Users reacted accordingly, providing strong negative results for these algorithms. Results from our 'typical' algorithms show some qualitative differences, but since users were exposed to two algorithms, the results may be biased. We present a wide variety of results, teasing out differences between algorithms. Finally, we succinctly summarize our most striking results as "Don't Look Stupid" in front of users.

Mcnee, S. M.: Meeting User Information Needs in Recommender Systems. Minneapolis, MN, USA, University of Minnesota, 2006

In order to build relevant, useful, and effective recommender systems, researchers need to understand why users come to these systems and how users judge recommendation lists. Today, researchers use accuracy-based metrics for judging goodness. Yet these metrics cannot capture users' criteria for judging recommendation usefulness. We need to rethink recommenders from a user's perspective: they help users find new information. Thus, not only do we need to know about the user, we need to know what the user is looking for. In this dissertation, we explore how to tailor recommendation lists not just to a user, but to the user's current information seeking task. We argue that each recommender algorithm has specific strengths and weaknesses, different from other algorithms. Thus, different recommender algorithms are better suited for specific users and their information seeking tasks. A recommender system should, then, select and tune the appropriate recommender algorithm (or algorithms) for a given user/information seeking task combination. To support this, we present results in three areas. First, we apply recommender systems in the domain of peer-reviewed computer science research papers, a domain where users have external criteria for selecting items to consume. The effectiveness of our approach is validated through several sets of experiments. Second, we argue that current recommender systems research in not focused on user needs, but rather on algorithm design and performance. To bring users back into focus, we reflect on how users perceive recommenders and the recommendation process, and present Human-Recommender Interaction theory, a framework and language for describing recommenders and the recommendation lists they generate. Third, we look to different ways of evaluating recommender systems algorithms. To this end, we propose a new set of recommender metrics, run experiments on several recommender algorithms using these metrics, and categorize the differences we discovered. Through Human-Recommender Interaction and these new metrics, we can bridge users and their needs with recommender algorithms to generate more useful recommendation lists.

Adomavicius, G. & Tuzhilin, A.: Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions. In: IEEE Transactions on Knowledge and Data Engineering 17 (2005), Nr. 6, S. 734-749

This paper presents an overview of the field of recommender systems and describes the current generation of recommendation methods that are usually classified into the following three main categories: content-based, collaborative, and hybrid recommendation approaches. This paper also describes various limitations of current recommendation methods and discusses possible extensions that can improve recommendation capabilities and make recommender systems applicable to an even broader range of applications. These extensions include, among others, an improvement of understanding of users and items, incorporation of the contextual information into the recommendation process, support for multcriteria ratings, and a provision of more flexible and less intrusive types of recommendations.

Herlocker, J. L.; Konstan, J. A.; Terveen, L. G. & Riedl, J. T.: Evaluating collaborative filtering recommender systems. In: ACM Trans. Inf. Syst. 22 (2004), S. 5-53
[Volltext]
Recommender systems have been evaluated in many, often incomparable, ways. In this article, we review the key decisions in evaluating collaborative filtering recommender systems: the user tasks being evaluated, the types of analysis and datasets being used, the ways in which prediction quality is measured, the evaluation of prediction attributes other than quality, and the user-based evaluation of the system as a whole. In addition to reviewing the evaluation strategies used by prior researchers, we present empirical results from the analysis of various accuracy metrics on one content domain where all the tested metrics collapsed roughly into three equivalence classes. Metrics within each equivalency class were strongly correlated, while metrics from different equivalency classes were uncorrelated.

Schein, A. I.; Popescul, A.; Ungar, L. H. & Pennock, D. M.: Methods and Metrics for Cold-start Recommendations. Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, NY, USA: ACM, 2002SIGIR '02 , S. 253-260
[Volltext]
We have developed a method for recommending items that combines content and collaborative data under a single probabilistic framework. We benchmark our algorithm against a naïve Bayes classifier on the cold-start problem, where we wish to recommend items that no one in the community has yet rated. We systematically explore three testing methodologies using a publicly available data set, and explain how these methods apply to specific real-world applications. We advocate heuristic recommenders when benchmarking to give competent baseline performance. We introduce a new performance metric, the CROC curve, and demonstrate empirically that the various components of our testing strategy combine to obtain deeper understanding of the performance characteristics of recommender systems. Though the emphasis of our testing is on cold-start recommending, our methods for recommending and evaluation are general.