From: John Sack <[log in to unmask]>
Date: Mon, 13 Oct 2014 05:49:53 -0700
I am forwarding this response on behalf of Anurag Acharya at Google
John Sack
Founding Director
HighWire Press
-----
I would like to clarify couple of things about our paper. My comments
are inline below,
cheers,
anurag
Corey Murata writes:
The basic flaw in the research is centered around how they identify
'elite journals.'
First, they are using incredibly broad disciplinary groupings from
Google Scholar Metrics:
http://scholar.google.com/citations?view_op=top_venues
Economics, for example is lumped in with Business and Management, and
if you look at the top ten journals in that broad group the only
management journal is MIS Quarterly, all the rest are Economics and
Finance.
[[ANURAG]] As described in the Methods section of the paper, elite
journals are identified for each of the 261 specific subject
categories (eg Immunology or Accounting & Taxation or Gender Studies
or Finance) and NOT at the level of broad areas (eg Health & Medical
Sciences or Business, Economics & Management).
To get an overview of changes within each broad area, we determined
the median, the 25th, and the 75th percentile subject categories
within each area. We then picked the median subject category in each
broad area as the representative for the area and plotted data for all
three of median/25th-percentile/75th-percentile categories in the
per-area graphs in Figure 2. The median/25th/75th percentile
categories were computed afresh for every year to ensure that they
remain representative of the area (details are in the Methods
section).
Second, they ignore the increase in the number and specialization of
journals over the period of the study. This increasing availability of
journals that are 'core' to a sub-disciplinary group of scholars would
naturally lead to more high-quality articles being published outside
of the 'elite' journals as defined by the authors of this paper. The
increasing number of journals also means that the ten 'elite' journals
becomes a progressively smaller percentage of the total scholarly
output over time.
[[ANURAG]] As mentioned above, the list of elite journals was computed
separately for each of the 261 specific subject categories. Which
means there are over 2500 journals that are considered elite each
year. As mentioned in the Methods section, the list of elite (and
non-elite) journals for each subject category was recomputed for each
year. So shifts in the focus of a subject category or new journals
that become a part of the "core" set would be reflected.
The Methods section of the paper also mentions that the number of
articles considered top-cited each year in a subject category was
fixed at 1000. Therefore, growth in the total number of articles
published isn't a significant factor. The top ten journals in a
subject category, as a group, publish more than 1000 articles per
year.
On Sun, Oct 12, 2014 at 11:08 AM, LIBLICENSE <[log in to unmask]> wrote:
>
> From: Corey Murata <[log in to unmask]>
> Date: Fri, 10 Oct 2014 10:48:20 -0700
>
> The basic flaw in the research is centered around how they identify
> 'elite journals.'
>
> First, they are using incredibly broad disciplinary groupings from
> Google Scholar Metrics:
>
> http://scholar.google.com/citations?view_op=top_venues
>
> Economics, for example is lumped in with Business and Management, and
> if you look at the top ten journals in that broad group the only
> management journal is MIS Quarterly, all the rest are Economics and
> Finance.
>
> Second, they ignore the increase in the number and specialization of
> journals over the period of the study. This increasing availability of
> journals that are 'core' to a sub-disciplinary group of scholars would
> naturally lead to more high-quality articles being published outside
> of the 'elite' journals as defined by the authors of this paper. The
> increasing number of journals also means that the ten 'elite' journals
> becomes a progressively smaller percentage of the total scholarly
> output over time.
>
> The final paragraph in the article states: "Now that finding and
> reading relevant articles in non-elite journals is about as easy as
> finding and reading articles in elite journals, researchers are
> increasingly building on and citing work published everywhere."
>
> This suggests that the findings they claim in the article are the
> result of changes in the discovery environment for scholarly
> literature despite the fact that there was nothing in their research
> that looked at changes in discovery as it relates to citation
> patterns.
>
> This typifies the essential problem (as I see it) with big data
> evangelism. It's easy to find patterns in data, but without context
> those patterns are meaningless, or worse, lead you to the wrong
> conclusion.
>
>
> Corey
> ****************
> Corey Murata
> Information Resources and Collection Assessment Librarian
> University of Washington Libraries
> Seattle, WA 98195
> [log in to unmask]
>
> On Thu, Oct 9, 2014 at 6:12 PM, LIBLICENSE <[log in to unmask]> wrote:
> >
> > From: Ann Shumelda Okerson <[log in to unmask]>
> > Date: Thu, 9 Oct 2014 20:27:07 -0400
> >
> > Via InfoDocket:
> >
> > A study by the Google Scholar team on the rise in importance of
> > non-elite journals has been deposited in arXiv. The abstract is
> > reproduced below. Any thoughts about the validity of the findings?
> > Do they take into account the overall growth of article publishing in
> > the time frame examined? What's really going on here? Ann
> >
> > *******
> >
> > http://www.infodocket.com/2014/10/08/new-research-from-google-rise-of-the-rest-the-growing-impact-of-non-elite-journals/
> >
> > In this paper, we examine the evolution of the impact of non-elite
> > journals. We attempt to answer two questions. First, what fraction of
> > the top-cited articles are published in non-elite journals and how has
> > this changed over time. Second, what fraction of the total citations
> > are to non-elite journals and how has this changed over time.
> >
> > We studied citations to articles published in 1995-2013. We computed
> > the 10 most-cited journals and the 1000 most-cited articles each year
> > for all 261 subject categories in Scholar Metrics. We marked the 10
> > most-cited journals in a category as the elite journals for the
> > category and the rest as non-elite.
> >
> > There are two conclusions from our study. First, the fraction of
> > top-cited articles published in non-elite journals increased steadily
> > over 1995-2013. While the elite journals still publish a substantial
> > fraction of high-impact articles, many more authors of well-regarded
> > papers in diverse research fields are choosing other venues.
> >
> > The number of top-1000 papers published in non-elite journals for the
> > representative subject category went from 149 in 1995 to 245 in 2013,
> > a growth of 64%. Looking at broad research areas, 4 out of 9 areas saw
> > at least one-third of the top-cited articles published in non-elite
> > journals in 2013. For 6 out of 9 areas, the fraction of top-cited
> > papers published in non-elite journals for the representative subject
> > category grew by 45% or more.
> >
> > Second, now that finding and reading relevant articles in non-elite
> > journals is about as easy as finding and reading articles in elite
> > journals, researchers are increasingly building on and citing work
> > published everywhere. Considering citations to all articles, the
> > percentage of citations to articles in non-elite journals went from
> > 27% in 1995 to 47% in 2013. Six out of nine broad areas had at least
> > 50% of citations going to articles published in non-elite journals in
> > 2013.
|
