Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

How do I parse a log file using only a shell script, for last one hour?

Each hour, the script should execute, parse the previous hour's logs, and the already alerted or listed errors should not repeat.

Error log example given below:

120622 13:06:36 mysqld_safe Starting mysqld daemon with databases from <path>
120622 13:06:36 [Note] Plugin'FEDERATED' is disabled.
120622 13:06:36  InnoDB: Initializing buffer pool, size = \78.0M
120622 13:06:36  InnoDB: Completed initialization of buffer pool
InnoDB: No valid checkpoint found.
InnoDB: If this error appears when you are creating an InnoDB database,
InnoDB: the problem may be that during an earlier attempt you managed
InnoDB: to create the InnoDB data files, but log file creation failed.
InnoDB: If that is the case, please refer to
InnoDB: http:<path>error-creating-innodb.html
120622 13:06:36 [ERROR] Plugin 'InnoDB' init function returned error.
120622 13:06:36 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
120622 13:06:36 [Note] Event Scheduler: Loaded 0 events
120622 13:06:36 [Note] <path>mysqld: ready for connections.

I want to alert only the ERROR d line.

share|improve this question

closed as not a real question by mgorven, Lucas Kauffman, Scott Pack, Ward, EEAA Aug 24 '12 at 3:54

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.If this question can be reworded to fit the rules in the help center, please edit the question.

Strikes me as a "write my script for me" question... at least as written, but probably in general too. – HopelessN00b Jul 19 '12 at 13:37
Rotate your logs hourly, run analysis on the individual log, which happen to be hour long. – Marcin Jul 19 '12 at 15:16

Check out logtail. It's a tool for reading lines in logs that have not been read.

share|improve this answer
I am looking for a shell script only. – ara Jul 19 '12 at 7:26
Since you need to know what's already read, you need to keep data about that in a state-file. I don't think it will be much overhead to use the logtail2-package, as implementing a shell script for this might give you more overhead. – pkhamre Jul 19 '12 at 7:59

Write the script like this and call it from cron every hour:



if [ -f $oldlogfile ]; then
  diff $oldlogfile $logfile | grep ERROR > /dev/null
  if [ $? -eq 0 ]; then
    diff $oldlogfile $logfile | grep ERROR | mail -s 'last hour mysql errors' $email
cp -p $logfile $oldlogfile
share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.