From: Joe Zucca <[log in to unmask]>
Date: Wed, 7 Oct 2015 11:08:37 -0400

Not an easy problem.  You might start by exploring Splunk http://www.splunk.com/

Joe Zucca
Director of Strategic Initiatives and Library Technology Services
University of Pennsylvania Libraries
Skype ID: jgz10101
orcid.org/0000-0002-6376-7810



Sent: Tuesday, October 06, 2015 4:46 PM
To: [log in to unmask]
Subject: Counting downloads in Apache log files
From: Daniel Feenberg <[log in to unmask]>

Date: Mon, 5 Oct 2015 16:24:37 -0400

We are a publisher and wish to count downloads by subscriber ip
address. Inspection of the Apache logfile is discouraging. Some
browsers seem to issue a request for the entire document (easy to
count, the return code in the log file is 200), some make multiple
requests for parital content (return code 206) and some do both for
the same document at nearly the same time. That is, first request the
entire document, and then a series of partial requests. I don't know
how common that is, but we can reproduce the behavior with Firefox at
least.

Can anyone refer us to either 1) software that can provide a proper
count or 2) information about how to process the log to obtain an
accurate count?

We have studied the "Counter" website and found software to analyze
counter-compatible logs, but no software to create them from standard
Apache logs. It appears that creating such software would be a major
undertaking.

Thanks for you attention

Daniel Feenberg
NBER