20

This is a basic question - in the movie "The Social Network" there are several scenes when the young Facebook staff is watching the PHP/Apache server logs on in their TERMINAL in real time.

I'm familiar with how to do this in a RUBY/RoR environment - but with a standard LAMP Apache/PHP environment, how to do actively monitor your server's actions in real time? I'm guessing there's an easy way to do this in Terminal.

2
  • I haven't seen this movie. Does anyone have a screenshot or a clip which shows the details of what the Facebook staff are doing in the movie? Oct 19, 2010 at 23:02
  • tail -f on the log file usually does it. Oct 17, 2013 at 1:35

12 Answers 12

50

maybe they use tail -f on the access log?

6
  • 2
    Wouldn't this get unmanageable really fast for large sites like Facebook where there are thousands of requests every second?
    – Vilx-
    Oct 19, 2010 at 9:49
  • 3
    tail -f can always be filtered through grep if you're looking for specific requests... that can reduce the volumes
    – Mark Baker
    Oct 19, 2010 at 9:58
  • 3
    tail -f is the no. 1 reason for putting at least minimal Cygwin on a windows box! Oct 19, 2010 at 10:21
  • 1
    @Daniel I think GnuWin32 also has a tail command
    – Jader Dias
    Oct 19, 2010 at 10:51
  • 1
    for that speed problem ... tail -f /log/any.log | while read line; do echo $line; sleep 3; done but this wouldn't be "realtime" anymore
    – theist
    Oct 19, 2010 at 13:58
18

Or even something like logstalgia ;-)

1
  • 3
    Wow that's awesome. And it might be more then just eye candy. Oct 19, 2010 at 23:39
8

or install and use multitail to specify filters and coloring :)

7

If you are looking for a real-time Apache log viewer and analyzer, I would definitely recommend GoAccess.

http://goaccess.prosoftcorp.com/

You just run it as (no conf needed):

goaccess -f /var/log/apache2/access.log -s -b
3

Another thing you can try is the "watch" command which can repetitively run a grep command on any log file you want. Using approriate watch command switches you can even highlight changes in log files in real time.

For example:

watch -d -n5 uptime

2

GoAccess 0.6.1 should be at least one of the tools in your bag. Open Source. Fast and STABLE.

For Rhel/ Centos 6.4

yum install glib2 glib2-devel glibc make geoip
yum install ncurses-devel
Download GoAccess - http://goaccess.prosoftcorp.com/download
Untar it and cd into it
./configure
make
make install

Finally: goaccess -f /var/log/httpd/access.log (or wherever your access logs are [maybe even /etc/httpd/logs/access_log]
1
1

This is a basic question - in the movie "The Social Network" there are several scenes when the young Facebook staff is watching the PHP/Apache server logs on in their TERMINAL in real time.

For a large system such as Facebook, you got to wonder how much artistic license (aka bullsh*t) was given in the making of the movie. In an environment with thousands of systems and thousands of clusters, watching server logs scrolling on a terminal is not necessarily the #1 monitoring activity.

In an environment like that (or any production environment with several dozen nodes), you'd have a NOSC of sort with indicators, and only when something is flagged as being in a state of crapping out that one would go and actively look at the logs (probably filtered for significant events and messages.)

2
  • For a small enough site, watching the logs is kind of fun, and can give you a general feel for how well your expectations are being met. In any case, the question just uses the movie as inspiration, asking "How can I do something like that?" instead of "How did they do that?"
    – eswald
    Oct 19, 2010 at 18:28
  • Well, one thing is to keep a terminal open with tail -f scrolling down. It could get very expensive bandwith-wide though. It can get very expensive CPU-wise at the client if you are, say, scrolling the logs on a remote terminal being displayed through VNC or Remote Desktop. Other solutions involve sending log output to syslog (and that having it send remotely to a syslog server), or even sending batches of log lines (compressed and via UDP) to a monitoring client. The later might involve loss of some lines since it's UDP. It might be acceptable in some conditions, though. Oct 19, 2010 at 19:59
0

While I'm thrilled that The Social Network used mostly accurate jargon and terminology, some of the stuff they did just for show.

I'm no server expert but I saw Zuckerberg have a terminal open with a ping command running and I can't think of any reason he would be randomly doing that. (This was after the time that they already had the server up and running).

I did find the Python server hacking scene very amusing though.

0

Logtool will also make nice colorized logs. Apache can write it's logs to a central NFS server, or can write logs using syslog, and syslog can send the log data to a central syslog server.

Click on the links for more detailed explanations.

tail -F /export/syslog/log/apache/access.log | logtool

0

You could also use the realtime log viewer in something like LogLogic and filter our what you did not want to see. This way you could collect more than apache logs, you could get router logs, ftp logs, etc.

0

you could write a polling script pretty easily, jsut compare the timestamp or the filesize and if changed, do a tail -n1

0

Apachetail also provides a real-time view/analysis of Apache logs by connecting directly to your Apache server from a windows desktop app. No software needs to be installed on the Apache server.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .