1

This is a high-level question regarding using goaccess as a tool for website analytics. I have it set up on an nginx server and parsing the logs just fine, but it seems that many requests to my own site have no Referer in the HTTP header. Is there a straightforward way of getting this information? Obviously I know that there are other sites with links to my own (such as a Github project or a HackerNews story) but they never show up in the goaccess referer information.

I also understand that it's possible that I'm completely misunderstanding and/or otherwise trying to stretch the limits of what is basically a simple access log viewer. That said, how does other analytics software get referrer information? Google Analytics, Piwik, and Mint all seem to do this fairly reliably.

EDIT- here are some lines from my log file. This is an instance where I know I entered the site from a Github.com link, which I would have expected to be shown as the referer. The immediate next line is where the style sheet is accessed, and it correctly identifies the referer as the site which accessed it.

69.139.121.104 - - [07/May/2013:18:57:46 -0400] "GET /2013/04/13/introducing-ratchet.html HTTP/1.1" 200 2195 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.29.13 (KHTML, like Gecko) Version/6.0.4 Safari/536.29.13"

69.139.121.104 - - [07/May/2013:18:57:46 -0400] "GET /css/styles.css HTTP/1.1" 200 1161 "http://micahmcfarland.com/2013/04/13/introducing-ratchet.html" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.29.13 (KHTML, like Gecko) Version/6.0.4 Safari/536.29.13"

EDIT2- I can't see referers because no referer information is sent between https and http. Not sure how I didn't know this but thanks a lot for reading and never mind!

Improve this question

1 Answer 1

Reset to default
1

I use GoAccess and it reports the referer field pretty well. I would say:

  • Make sure nginx is recording the referer data in your log, otherwise goaccess won't report it
  • Make sure you have the right log format string configured in your ~/.goaccessrc.

Keep in mind that the referer is sent by the client's browser, and is therefore unreliable. It might not be there or it might be forged.

Also, can you post a couple of lines from your nginx log that are not being reported and contain referer?

Google Analytics does not rely in a log file.

Improve this answer
3
  • I've updated the original question with some lines from the log file. Thanks for your help!
    – Micah
    May 7, 2013 at 23:44
  • I get http://micahmcfarland.com/2013/04/13/introducing-ratchet.html and - in the referer report. What output do you get when parsing those lines with goaccess?
    – Kayla
    May 8, 2013 at 0:38
  • Thanks a lot for your help -- turns out, I'm missing referer information because it isn't sent between https --> http connections. I suppose it would help me to learn the basics of the internet before coming to ServerFault :) Thanks again though!
    – Micah
    May 15, 2013 at 1:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .