4

I am trying to set up a python script to query a DB for all updates over the past 5 minutes. Given the number of updates, it must be as precise as possible (our records are timestamped to the microsecond). There are two ways I'm considering tackling this - first is using cron. However this relies on cron being precise enough to always run at the exact same interval every time it runs. So if the first execution time is 00:00:00.123456789, it would need to run again at 00:05:00.123456789. Otherwise, there is a possibility of records being missed in the gaps in between.

The other option is finding a way to "snap" the sql query to the nearest minute, rounded down. But if I can use cron, i'd prefer to do that to keep things as simple as possible.

1 Answer 1

3

I've had cron scripts that for some reason or another, needed to output the time and I have seen them sometimes be a second late (only ever tracked down to the second). I do not know if this is due to Cron or due to the variations in how long it takes to load and execute the script. I imagine it is a little of both. Either way, relying on the sql query to execute at the exact same microsecond, is not going to work.

You can have your python script get the current time and then round it the closest 5-min interval. For example, if it retrieves the current time as 00:05:03.123, just drop the seconds and query based on 00:05:00 and back to 00:00:00.

Either that or you could record the timestamp of the last record, and then next time query from the current time back to that timestamp.

Edit: removed the first sentence of the second paragraph - it didn't really make sense there.

3
  • Thanks James! That was my second option :) I've never tried relying on cron for time-sensitive things before so I just wanted to see if it would meet my needs.
    – Matthew
    May 23, 2011 at 13:13
  • You'll never get a non-realtime operating system to do cron with the precision you require, unless loads and other factors are incredibly predictable and consistent. Some other option (like James suggests) would be required. May 23, 2011 at 13:19
  • @Matthew ah, I see now that is what you meant by "snap" the sql query to the nearest minute. You would still use cron to run the python script though.
    – James
    May 23, 2011 at 15:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .