Email is an increasingly important and ubiquitous means of communication, both facilitating contact between private individuals and enabling rises in the productivity of organizations. However the relentless rise of automatic unauthorized emails, a.k.a. spam is eroding away much of the attractiveness of email communication. Most of the attention dedicated to date to spam detection has focused on the content of the emails or on the addresses or domains associated with spam senders. Although methods based on these - easily changeable - identifiers work reasonably well they miss on the fundamental nature of spam as an opportunistic relationship, very different from the normal mutual relations between senders and recipients of legitimate email. Here we present a comprehensive graph theoretical analysis of email traffic that captures these properties quantitatively. We identify several simple metrics that serve both to distinguish between spam and legitimate email and to provide a statistical basis for models of spam traffic.

@misc{gomes-2005, author = {Gomes, Luiz H. and Almeida, Rodrigo B. and Bettencourt, Luis M. A. and Almeida, Virgilio and Almeida, Jussara M.}, title = {Comparative Graph Theoretical Characterization of Networks of Spam and Legitimate Email}, year = {2005}, url = {http://www.citebase.org/abstract?id=oai:arXiv.org:physics/0504025}, keywords = {email, graph, network, spam}, abstract = {Email is an increasingly important and ubiquitous means of communication, both facilitating contact between private individuals and enabling rises in the productivity of organizations. However the relentless rise of automatic unauthorized emails, a.k.a. spam is eroding away much of the attractiveness of email communication. Most of the attention dedicated to date to spam detection has focused on the content of the emails or on the addresses or domains associated with spam senders. Although methods based on these - easily changeable - identifiers work reasonably well they miss on the fundamental nature of spam as an opportunistic relationship, very different from the normal mutual relations between senders and recipients of legitimate email. Here we present a comprehensive graph theoretical analysis of email traffic that captures these properties quantitatively. We identify several simple metrics that serve both to distinguish between spam and legitimate email and to provide a statistical basis for models of spam traffic.} }

%0 = misc %A = Gomes, Luiz H. and Almeida, Rodrigo B. and Bettencourt, Luis M. A. and Almeida, Virgilio and Almeida, Jussara M. %B = } %C = %D = 2005 %I = %T = Comparative Graph Theoretical Characterization of Networks of Spam and Legitimate Email} %U = http://www.citebase.org/abstract?id=oai:arXiv.org:physics/0504025

<sec>
itle>Background</title>
>Intricate maps of science have been created from citation data to visualize the structure of scientific activity. However, most scientific publications are now accessed online. Scholarly web portals record detailed log data at a scale that exceeds the number of all existing citations combined. Such log data is recorded immediately upon publication and keeps track of the sequences of user requests (clickstreams) that are issued by a variety of users across many different domains. Given these advantages of log datasets over citation data, we investigate whether they can produce high-resolution, more current maps of science.</p>
sec><sec>
itle>Methodology</title>
>Over the course of 2007 and 2008, we collected nearly 1 billion user interactions recorded by the scholarly web portals of some of the most significant publishers, aggregators and institutional consortia. The resulting reference data set covers a significant part of world-wide use of scholarly web portals in 2006, and provides a balanced coverage of the humanities, social sciences, and natural sciences. A journal clickstream model, i.e. a first-order Markov chain, was extracted from the sequences of user interactions in the logs. The clickstream model was validated by comparing it to the Getty Research Institute's Architecture and Art Thesaurus. The resulting model was visualized as a journal network that outlines the relationships between various scientific domains and clarifies the connection of the social sciences and humanities to the natural sciences.</p>
sec><sec>
itle>Conclusions</title>
>Maps of science resulting from large-scale clickstream data provide a detailed, contemporary view of scientific activity and correct the underrepresentation of the social sciences and humanities that is commonly found in citation data.</p>
sec>

@article{bollen2009clickstream, author = {Bollen, Johan and Van de Sompel, Herbert and Hagberg, Aric and Bettencourt, Luis and Chute, Ryan and Rodriguez, Marko A. and Balakireva, Lyudmila}, title = {Clickstream Data Yields High-Resolution Maps of Science}, journal = {PLoS ONE}, publisher = {Public Library of Science}, year = {2009}, volume = {4}, number = {3}, pages = {e4803}, url = {http://dx.doi.org/10.1371%2Fjournal.pone.0004803}, doi = {10.1371/journal.pone.0004803}, keywords = {navigation, markov, clickstream, path}, abstract = {<sec><title>Background</title><p>Intricate maps of science have been created from citation data to visualize the structure of scientific activity. However, most scientific publications are now accessed online. Scholarly web portals record detailed log data at a scale that exceeds the number of all existing citations combined. Such log data is recorded immediately upon publication and keeps track of the sequences of user requests (clickstreams) that are issued by a variety of users across many different domains. Given these advantages of log datasets over citation data, we investigate whether they can produce high-resolution, more current maps of science.</p></sec><sec><title>Methodology</title><p>Over the course of 2007 and 2008, we collected nearly 1 billion user interactions recorded by the scholarly web portals of some of the most significant publishers, aggregators and institutional consortia. The resulting reference data set covers a significant part of world-wide use of scholarly web portals in 2006, and provides a balanced coverage of the humanities, social sciences, and natural sciences. A journal clickstream model, i.e. a first-order Markov chain, was extracted from the sequences of user interactions in the logs. The clickstream model was validated by comparing it to the Getty Research Institute's Architecture and Art Thesaurus. The resulting model was visualized as a journal network that outlines the relationships between various scientific domains and clarifies the connection of the social sciences and humanities to the natural sciences.</p></sec><sec><title>Conclusions</title><p>Maps of science resulting from large-scale clickstream data provide a detailed, contemporary view of scientific activity and correct the underrepresentation of the social sciences and humanities that is commonly found in citation data.</p></sec>} }

%0 = article %A = Bollen, Johan and Van de Sompel, Herbert and Hagberg, Aric and Bettencourt, Luis and Chute, Ryan and Rodriguez, Marko A. and Balakireva, Lyudmila %D = 2009 %I = Public Library of Science %T = Clickstream Data Yields High-Resolution Maps of Science %U = http://dx.doi.org/10.1371%2Fjournal.pone.0004803

Background Intricate maps of science have been created from citation data to visualize the structure of scientific activity. However, most scientific publications are now accessed online. Scholarly web portals record detailed log data at a scale that exceeds the number of all existing citations combined. Such log data is recorded immediately upon publication and keeps track of the sequences of user requests (clickstreams) that are issued by a variety of users across many different domains. Given these advantages of log datasets over citation data, we investigate whether they can produce high-resolution, more current maps of science. Methodology Over the course of 2007 and 2008, we collected nearly 1 billion user interactions recorded by the scholarly web portals of some of the most significant publishers, aggregators and institutional consortia. The resulting reference data set covers a significant part of world-wide use of scholarly web portals in 2006, and provides a balanced coverage of the humanities, social sciences, and natural sciences. A journal clickstream model, i.e. a first-order Markov chain, was extracted from the sequences of user interactions in the logs. The clickstream model was validated by comparing it to the Getty Research Institute's Architecture and Art Thesaurus. The resulting model was visualized as a journal network that outlines the relationships between various scientific domains and clarifies the connection of the social sciences and humanities to the natural sciences. Conclusions Maps of science resulting from large-scale clickstream data provide a detailed, contemporary view of scientific activity and correct the underrepresentation of the social sciences and humanities that is commonly found in citation data.

@article{bollen2009clickstream, author = {Bollen, Johan and van de Sompel, Herbert and Hagberg, Aric and Bettencourt, Luis and Chute, Ryan and Rodriguez, Marko A. and Balakireva, Lyudmila}, title = {Clickstream Data Yields High-Resolution Maps of Science}, journal = {PLoS ONE}, publisher = {Public Library of Science}, year = {2009}, volume = {4}, number = {3}, pages = {e4803}, url = {http://dx.doi.org/10.1371/journal.pone.0004803}, doi = {10.1371/journal.pone.0004803}, keywords = {scientometrics, science, analysis, citation, toread}, abstract = {Background Intricate maps of science have been created from citation data to visualize the structure of scientific activity. However, most scientific publications are now accessed online. Scholarly web portals record detailed log data at a scale that exceeds the number of all existing citations combined. Such log data is recorded immediately upon publication and keeps track of the sequences of user requests (clickstreams) that are issued by a variety of users across many different domains. Given these advantages of log datasets over citation data, we investigate whether they can produce high-resolution, more current maps of science. Methodology Over the course of 2007 and 2008, we collected nearly 1 billion user interactions recorded by the scholarly web portals of some of the most significant publishers, aggregators and institutional consortia. The resulting reference data set covers a significant part of world-wide use of scholarly web portals in 2006, and provides a balanced coverage of the humanities, social sciences, and natural sciences. A journal clickstream model, i.e. a first-order Markov chain, was extracted from the sequences of user interactions in the logs. The clickstream model was validated by comparing it to the Getty Research Institute's Architecture and Art Thesaurus. The resulting model was visualized as a journal network that outlines the relationships between various scientific domains and clarifies the connection of the social sciences and humanities to the natural sciences. Conclusions Maps of science resulting from large-scale clickstream data provide a detailed, contemporary view of scientific activity and correct the underrepresentation of the social sciences and humanities that is commonly found in citation data.} }

%0 = article %A = Bollen, Johan and van de Sompel, Herbert and Hagberg, Aric and Bettencourt, Luis and Chute, Ryan and Rodriguez, Marko A. and Balakireva, Lyudmila %D = 2009 %I = Public Library of Science %T = Clickstream Data Yields High-Resolution Maps of Science %U = http://dx.doi.org/10.1371/journal.pone.0004803