Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

This is a high-level question regarding using goaccess as a tool for website analytics. I have it set up on an nginx server and parsing the logs just fine, but it seems that many requests to my own site have no Referer in the HTTP header. Is there a straightforward way of getting this information? Obviously I know that there are other sites with links to my own (such as a Github project or a HackerNews story) but they never show up in the goaccess referer information.

I also understand that it's possible that I'm completely misunderstanding and/or otherwise trying to stretch the limits of what is basically a simple access log viewer. That said, how does other analytics software get referrer information? Google Analytics, Piwik, and Mint all seem to do this fairly reliably.

EDIT- here are some lines from my log file. This is an instance where I know I entered the site from a Github.com link, which I would have expected to be shown as the referer. The immediate next line is where the style sheet is accessed, and it correctly identifies the referer as the site which accessed it.

69.139.121.104 - - [07/May/2013:18:57:46 -0400] "GET /2013/04/13/introducing-ratchet.html HTTP/1.1" 200 2195 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.29.13 (KHTML, like Gecko) Version/6.0.4 Safari/536.29.13"

69.139.121.104 - - [07/May/2013:18:57:46 -0400] "GET /css/styles.css HTTP/1.1" 200 1161 "http://micahmcfarland.com/2013/04/13/introducing-ratchet.html" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.29.13 (KHTML, like Gecko) Version/6.0.4 Safari/536.29.13"

EDIT2- I can't see referers because no referer information is sent between https and http. Not sure how I didn't know this but thanks a lot for reading and never mind!

share|improve this question

1 Answer 1

I use GoAccess and it reports the referer field pretty well. I would say:

  • Make sure nginx is recording the referer data in your log, otherwise goaccess won't report it
  • Make sure you have the right log format string configured in your ~/.goaccessrc.

Keep in mind that the referer is sent by the client's browser, and is therefore unreliable. It might not be there or it might be forged.

Also, can you post a couple of lines from your nginx log that are not being reported and contain referer?

Google Analytics does not rely in a log file.

share|improve this answer
    
I've updated the original question with some lines from the log file. Thanks for your help! –  Micah May 7 '13 at 23:44
    
I get http://micahmcfarland.com/2013/04/13/introducing-ratchet.html and - in the referer report. What output do you get when parsing those lines with goaccess? –  Kayla May 8 '13 at 0:38
    
Thanks a lot for your help -- turns out, I'm missing referer information because it isn't sent between https --> http connections. I suppose it would help me to learn the basics of the internet before coming to ServerFault :) Thanks again though! –  Micah May 15 '13 at 1:27

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.