Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have multiple log files of 10mb each. The content of the log files always look like this:

06.09.2012, 21:36:02:83: xxxxx yyyyyy zzz

06.09.2012, 21:36:02:84: xx yyy zzz

06.09.2012, 21:36:05:24: xxxxxxxxx yyyyyyyyyy zzzz

The log files are in the same folder and have the following names File01.log, File02.log, etc.

I want to create a new log file for every hour. This new log file can contain lines from multiple log files, since a file can end at 21:36:05 because it reached the 10 mb. The next logfile will then continue from that point of time. It is thus very random at what time a log file ends. One thing however is for sure. File02.log always follows up File01 and so further.

My idea was to execute it in a .bat, using a for /f loop to loop through all files and parse out the date and time from the logging. I wanted to convert the date and time to seconds and compare this to an hour interval (which is also in seconds). If the time in seconds is bigger than the interval, the interval should be increased with an hour (both lower and upper bound).

I had something in mind like the following code. I however have not much experience in programming so I am not sure if this is possible. Perhaps one of you guys can give me a push in the right direction or provide me with a solution.

FOR /f "tokens=1,2 delims= " %%G in ('dir /O:N /b/s ^"C:\logs\R\*.log^"') DO 

while (records in the current logfile stille exist) ::insert a condition which checks whether the end of the log file is not reached yet

DO

DateFromRecord = %%G :: parse out the date
TimeFromRecord = %%H :: parse out the time
:: Make a variable that combines both the date and the time to a timestamp
:: Create a function that turns the timestamp into seconds

if (TimestampInSeconds ge lowerbound AND TimestampInSeconds lt higherbound) ::compare the time from file with the interval (in seconds)
then
copy line to a new log file

elseif (TimestampInSeconds ge higherbound) :: if higher than the interval, increase the interval with a day and 
then 
lowerbound(loweverbound + aantal seconden van 1dag)
higherbound(higherbound + aantal seconden van 1dag)

done
done
share|improve this question
1  
I'm not sure that bash means what you think it means. bash is a unix shell, what you have there looks like a windows batch file. –  DerfK Oct 30 '12 at 9:45
    
If this really is Linux, syslog-ng can start a new log file every hour simply by putting the $HOUR variable in the file name. I'm sure rsyslog can do something similar. This would also match your filename pattern requirement as long as you don't mind it starting from 00 again each day. –  Ladadadada Oct 30 '12 at 10:00
    
You are right DerfK, I was a bit confused... I am completely new into creating .bat files that can execute a procedure like this. Hope someone can help me out! –  Larss Oct 30 '12 at 10:18
    
Doing this kind of stuff with .bat files is only for those who also like whips and chains. OTOH, it's easy in Perl and similar languages. –  John Gardeniers Oct 30 '12 at 11:04

1 Answer 1

I think you are overcomplicating it. Instead of doing the math and such, just use what you already have at your disposal... you already have a line that has the date and time, so just create a log file for each hour using that info.

The following code should be close to what you need... essentially, for each log file that is found it will type out each line, breaking it into the first four pieces (month, day, year, hour) and then the rest in the fifth variable.

Then it just echos that line out to a logfile named "YYYY-MM-DD HH00.log"

So, for your example lines, they would be put into the log file "2012-06-09 2100.log" that would contain all of the lines from the 9pm hour on June 9th, 2012.

Here is the code I came up with:

@Echo off
rem Loop through the files, in name order, and call the below routine
rem passing the filename in quotes (in case of spaces)
rem 
FOR /f "tokens=1 delims= " %%G in ('dir /O:N /b/s ^"C:\logs\R\*.log^"') DO call :ProcessFile "%%G"

rem When we get to this point we have processed all the files.
rem skip to the end to avoid processing the subroutine with no input
Goto :eof

:ProcessFile
rem Remove the quotation marks from around the filename
Set @fileName=%~1

rem Type out the file contents.  
rem Break at delimiters of comma, period, space, or colon
rem Given a sample line of this input:
rem      06.09.2012, 21:36:02:83: xxxxx yyyyyy zzz
rem (and assuming US Date order of MM.DD.YYYY)
rem you will end up with the following variables:
rem     %l - Month - 06
rem     %m - Day - 09
rem     %n - Year - 2012
rem     %o - Hour - 21
rem     %p - everything else - 36:02:83: xxxxx yyyyyy zzz
rem
rem Assuming that the user needs the entire line output to the new files,
rem we need to reconstruct the original line, replacing the delimiters that were removed.
rem So we need to echo the following:
rem     %%l.%%m.%%n, %%o:%%p
rem and the output filename is the new log file name:
rem     %%n-%%l-%%m %%o00.log
rem
rem
for /f "tokens=1,2,3,4,* delims=:,. " %%l in ('type ^"%@fileName%^"') do echo %%l.%%m.%%n, %%o:%%p >> "%%n-%%l-%%m %%o00.log"

:eof
rem End of program
share|improve this answer
    
Set up a test that mimics your environment and this seemed to work... –  Glenn Sullivan Oct 30 '12 at 13:04

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.