From: Corey Murata <[log in to unmask]>
Date: Fri, 10 Oct 2014 10:48:20 -0700

The basic flaw in the research is centered around how they identify
'elite journals.'

First, they are using incredibly broad disciplinary groupings from
Google Scholar Metrics:

http://scholar.google.com/citations?view_op=top_venues

Economics, for example is lumped in with Business and Management, and
if you look at the top ten journals in that broad group the only
management journal is MIS Quarterly, all the rest are Economics and
Finance.

Second, they ignore the increase in the number and specialization of
journals over the period of the study. This increasing availability of
journals that are 'core' to a sub-disciplinary group of scholars would
naturally lead to more high-quality articles being published outside
of the 'elite' journals as defined by the authors of this paper. The
increasing number of journals also means that the ten 'elite' journals
becomes a progressively smaller percentage of the total scholarly
output over time.

The final paragraph in the article states:  "Now that finding and
reading relevant articles in non-elite journals is about as easy as
finding and reading articles in elite journals, researchers are
increasingly building on and citing work published everywhere."

This suggests that the findings they claim in the article are the
result of changes in the discovery environment for scholarly
literature despite the fact that there was nothing in their research
that looked at changes in discovery as it relates to citation
patterns.

This typifies the essential problem (as I see it) with big data
evangelism. It's easy to find patterns in data, but without context
those patterns are meaningless, or worse, lead you to the wrong
conclusion.


Corey
****************
Corey Murata
Information Resources and Collection Assessment Librarian
University of Washington Libraries
Seattle, WA 98195
[log in to unmask]

On Thu, Oct 9, 2014 at 6:12 PM, LIBLICENSE <[log in to unmask]> wrote:
>
> From: Ann Shumelda Okerson <[log in to unmask]>
> Date: Thu, 9 Oct 2014 20:27:07 -0400
>
> Via InfoDocket:
>
> A study by the Google Scholar team on the rise in importance of
> non-elite journals has been deposited in arXiv.  The abstract is
> reproduced below.  Any thoughts about the validity of the findings?
> Do they take into account the overall growth of article publishing in
> the time frame examined?   What's really going on here?  Ann
>
> *******
>
> http://www.infodocket.com/2014/10/08/new-research-from-google-rise-of-the-rest-the-growing-impact-of-non-elite-journals/
>
> In this paper, we examine the evolution of the impact of non-elite
> journals. We attempt to answer two questions. First, what fraction of
> the top-cited articles are published in non-elite journals and how has
> this changed over time. Second, what fraction of the total citations
> are to non-elite journals and how has this changed over time.
>
> We studied citations to articles published in 1995-2013. We computed
> the 10 most-cited journals and the 1000 most-cited articles each year
> for all 261 subject categories in Scholar Metrics. We marked the 10
> most-cited journals in a category as the elite journals for the
> category and the rest as non-elite.
>
> There are two conclusions from our study. First, the fraction of
> top-cited articles published in non-elite journals increased steadily
> over 1995-2013. While the elite journals still publish a substantial
> fraction of high-impact articles, many more authors of well-regarded
> papers in diverse research fields are choosing other venues.
>
> The number of top-1000 papers published in non-elite journals for the
> representative subject category went from 149 in 1995 to 245 in 2013,
> a growth of 64%. Looking at broad research areas, 4 out of 9 areas saw
> at least one-third of the top-cited articles published in non-elite
> journals in 2013. For 6 out of 9 areas, the fraction of top-cited
> papers published in non-elite journals for the representative subject
> category grew by 45% or more.
>
> Second, now that finding and reading relevant articles in non-elite
> journals is about as easy as finding and reading articles in elite
> journals, researchers are increasingly building on and citing work
> published everywhere. Considering citations to all articles, the
> percentage of citations to articles in non-elite journals went from
> 27% in 1995 to 47% in 2013. Six out of nine broad areas had at least
> 50% of citations going to articles published in non-elite journals in
> 2013.