up vote 3 down vote favorite
1
share [g+] share [fb]

This is a basic question - in the movie "The Social Network" there are several scenes when the young Facebook staff is watching the PHP/Apache server logs on in their TERMINAL in real time.

I'm familiar with how to do this in a RUBY/RoR environment - but with a standard LAMP Apache/PHP environment, how to do actively monitor your server's actions in real time? I'm guessing there's an easy way to do this in Terminal.

link|improve this question
I haven't seen this movie. Does anyone have a screenshot or a clip which shows the details of what the Facebook staff are doing in the movie? – Stefan Lasiewski Oct 19 '10 at 23:02
feedback

migrated from stackoverflow.com Oct 19 '10 at 9:46

This question came from our site for professional and enthusiast programmers.

11 Answers

up vote 32 down vote

maybe they use tail -f on the access log?

link|improve this answer
Wouldn't this get unmanageable really fast for large sites like Facebook where there are thousands of requests every second? – Vilx- Oct 19 '10 at 9:49
2  
tail -f can always be filtered through grep if you're looking for specific requests... that can reduce the volumes – Mark Baker Oct 19 '10 at 9:58
1  
tail -f is the no. 1 reason for putting at least minimal Cygwin on a windows box! – Daniel Earwicker Oct 19 '10 at 10:21
1  
@Daniel I think GnuWin32 also has a tail command – Jader Dias Oct 19 '10 at 10:51
1  
for that speed problem ... tail -f /log/any.log | while read line; do echo $line; sleep 3; done but this wouldn't be "realtime" anymore – theist Oct 19 '10 at 13:58
show 1 more comment
feedback
up vote 10 down vote

Or even something like logstalgia ;-)

link|improve this answer
2  
Wow that's awesome. And it might be more then just eye candy. – Stefan Lasiewski Oct 19 '10 at 23:39
feedback
up vote 3 down vote

or install and use multitail to specify filters and coloring :)

link|improve this answer
feedback
up vote 1 down vote

This is a basic question - in the movie "The Social Network" there are several scenes when the young Facebook staff is watching the PHP/Apache server logs on in their TERMINAL in real time.

For a large system such as Facebook, you got to wonder how much artistic license (aka bullsh*t) was given in the making of the movie. In an environment with thousands of systems and thousands of clusters, watching server logs scrolling on a terminal is not necessarily the #1 monitoring activity.

In an environment like that (or any production environment with several dozen nodes), you'd have a NOSC of sort with indicators, and only when something is flagged as being in a state of crapping out that one would go and actively look at the logs (probably filtered for significant events and messages.)

link|improve this answer
For a small enough site, watching the logs is kind of fun, and can give you a general feel for how well your expectations are being met. In any case, the question just uses the movie as inspiration, asking "How can I do something like that?" instead of "How did they do that?" – eswald Oct 19 '10 at 18:28
Well, one thing is to keep a terminal open with tail -f scrolling down. It could get very expensive bandwith-wide though. It can get very expensive CPU-wise at the client if you are, say, scrolling the logs on a remote terminal being displayed through VNC or Remote Desktop. Other solutions involve sending log output to syslog (and that having it send remotely to a syslog server), or even sending batches of log lines (compressed and via UDP) to a monitoring client. The later might involve loss of some lines since it's UDP. It might be acceptable in some conditions, though. – Luis Espinal Oct 19 '10 at 19:59
feedback
up vote 0 down vote

you could write a polling script pretty easily, jsut compare the timestamp or the filesize and if changed, do a tail -n1

link|improve this answer
feedback
up vote 0 down vote

While I'm thrilled that The Social Network used mostly accurate jargon and terminology, some of the stuff they did just for show.

I'm no server expert but I saw Zuckerberg have a terminal open with a ping command running and I can't think of any reason he would be randomly doing that. (This was after the time that they already had the server up and running).

I did find the Python server hacking scene very amusing though.

link|improve this answer
feedback
up vote 0 down vote

Logtool will also make nice colorized logs. Apache can write it's logs to a central NFS server, or can write logs using syslog, and syslog can send the log data to a central syslog server.

Click on the links for more detailed explanations.

tail -F /export/syslog/log/apache/access.log | logtool

link|improve this answer
feedback
up vote 0 down vote

You could also use the realtime log viewer in something like LogLogic and filter our what you did not want to see. This way you could collect more than apache logs, you could get router logs, ftp logs, etc.

link|improve this answer
feedback
up vote 0 down vote

If you are looking for a real-time Apache log viewer and analyzer, I would definitely recommend GoAccess.

http://goaccess.prosoftcorp.com/

You just run it as (no conf needed):

goaccess -f /var/log/apache2/access.log -s -b
link|improve this answer
feedback
up vote -1 down vote

@Stefan Lasiewski

I tried http://goaccess.prosoftcorp.com/ but it's giving an error "Your terminal does not support color" - can this only be run from the physical machine? I'm using SSH at the moment and tried various clients.

Fun idea though. I have a 50" monitoring screen that it would be amusing to view the access logs in real time rather than the drab 5min uptime checks that normal goes on!

link|improve this answer
@user57833 - The user @DanMor21 pointed you to goaccess, not I. – Stefan Lasiewski Nov 17 '10 at 18:08
feedback
up vote -1 down vote

Is this for Linux? Anything like for Windows XAMPP? :D

link|improve this answer
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.