Preferred format of file names which include a timestamp

Question

As we all know "unix" can have anything in a file except '/' and '\0', sysadmins however tend to have a much smaller preference, mainly due to nothing liking spaces as input ... and a bunch of things having a special meaning for ':' and '@' among others.

Recently I'd seen yet another case where a timestamp was used in a filename, and after playing with different formats a bit to make it "better" I figured I'd try to find a "best practice", not seeing one I figured I'd just ask here and see what people thought.

Possible "common" solutions (p=prefix and s=suffix):

1. syslog/logrotate/DNS like format:

   p-%Y%m%d-suffix = prefix-20110719-s
   p-%Y%m%d%H%M-suffix = prefix-201107191732-s
   p-%Y%m%d%H%M%S-suffix = prefix-20110719173216-s
   

   pros:

   • It's "common", so "good enough" might be better than "best".
   • No weird characters.
   • Easy to distinguish the "date/time blob" from everything else.

   cons:

   • The date only version isn't easy to read, and including the time makes my eyes bleed and seconds as well is just "lol".
   • Assumes TZ.

2. ISO-8601- format

   p-%Y-%m-%d-s = p-2011-07-19-s
   p-%Y-%m-%dT%H:%M%z-s = p-2011-07-19T17:32-0400-s
   p-%Y-%m-%dT%H:%M:%S%z-s = p-2011-07-19T17:32:16-0400-s
   p-%Y-%m-%dT%H:%M:%S%z-s = p-2011-07-19T23:32:16+0200-s
   

   pros:

   • No spaces.
   • Takes TZ into account.
   • Is "not bad" to read by humans (date only is v. good).
   • Can be generated by $(date --iso={hours,minutes,seconds})

   cons:

   • scp/tar/etc. won't like those ':' characters.
   • Takes a bit for "normal" people to see WTF that 'T' is for, and what the thing at the end is :).
   • Lots of '-' characters.

3. rfc-3339 format

   p-%Y-%m-%d-s = p-2011-07-19-s
   p-%Y-%m-%d %H:%M%:z-s = p-2011-07-19 17:32-04:00-s
   p-%Y-%m-%d %H:%M:%S%:z-s = p-2011-07-19 17:32:16-04:00-s
   p-%Y-%m-%d %H:%M:%S%:z-s = p-2011-07-19 23:32:16+02:00-s
   

   pros:

   • Takes TZ into account.
   • Can easily be read by "all humans".
   • Can distinguish date/time from prefix/suffix.
   • Some of the above can be generated with $(date --iso={hours,seconds})

   cons:

   • Has spaces in the time versions (which means all code will hate it).
   • scp/tar/etc. won't like those ':' characters.

4. I love hyphens:

   p-%Y-%m-%d-s = p-2011-07-19-s
   p-%Y-%m-%d-%H-%M-s = p-2011-07-19-17-32-s
   p-%Y-%m-%d-%H-%M-%S-s = p-2011-07-19-23-32-16-s
   

   pros:

   • basically a slightly nicer syslog/etc. variant.

   cons:

   • Lots of '-' characters.
   • Assumes TZ.

5. I love hyphens, with extensions:

   p.%Y-%m-%d.s = p.2011-07-19.s
   p.%Y-%m-%d.%H-%M.s = p.2011-07-19.17-32.s
   p.%Y-%m-%d.%H-%M-%S.s = p.2011-07-19.23-32-16.s
   

   pros:

   • basically a slightly nicer "I love hyphens" variant.
   • No weird characters.
   • Can distinguish date/time from prefix/suffix.

   cons:

   • Using '.' here is somewhat non-traditional.
   • Assumes TZ.

...so anyone want to give a preference and a reason, or more than one (Eg. don't care about TZ if it's 95+% to stay machine local, but care a lot if it isn't).

Or, obviously, something not in the above list.

Answer 1

1. ISO 8601 format should be adhered to as much as possible, since it is the closest thing there is to a standard.
2. The 'T' is not enough of a stumbling block to really warrant getting rid of it.
3. The ':'s are potentially killers, so those should be avoided.
4. For the reasons mentioned in others' answers, UTC (or 'Z' time) should be used.
5. ISO 8601 includes a format using UTC ('Z' time), which should be used.
6. ISO 8601 includes a format that does not use the ':' character, which should be used.

---

So...sample 'best' date-time formats:

1. 20120317T1748Z

   • 100% in accordance with ISO 8601
   • alphanumeric characters only (very sysadmin-friendly)
   • not the quickest to read, but certainly readable by the layperson

2. 2012-03-17T1748Z

   • date portion is in accordance with ISO 8601
   • time portion is in accordance with ISO 8601
   • transition between date and time is in accordance with ISO 8601
   • mixes the ISO 8601 'extended' format (date with hyphens, time with colons) with the ISO 8601 'basic' format (date without hyphens, time without colons), which is likely not quite right
   • adds '-' character (vs 1.)
   • a bit easier for the layperson to read (vs 1.)

3. 2012-03-17--1748Z

   • date portion is in accordance with ISO 8601
   • time portion is in accordance with ISO 8601
   • transition between date and time is not in accordance with ISO 8601
   • mixes the ISO 8601 'extended' format with the ISO 8601 'basic' format
   • a bit easier for the layperson to read (vs 1. and 2.)
   • no new characters (vs 2.)

I am partial to 1. since it is fully IAW the standard, but the others are close.

Note:: Add seconds as necessary, of course. ...and yes, with or without seconds (or even minutes) is all IAW ISO 8601. :)

Answer 2

I would not include a time zone, only use universal time. If there may be confusion, you could add a -UTC suffix. If you specify a time zone, someone may depend on it. And there would be weird edge cases where DST changes or DST shifts wreak havoc on some processing, or processing is different on some systems because their DST configuration is not up to date. UTC is always the same everywhere.

I do think that hyphens make the file name more readable, in the sense that it makes it easier to discern the datetime of the file data. If you want to include sub-second precision, that usually is .nnnnn.

I personally don't like the T. Using a colon in a file name may affect interoperability with other file systems.

Answer 3

For whatever it's worth, this is the format that the screenshot utility scrot uses by default for the filename (i.e.: when no filename is specified):

%Y-%m-%d-%H%M%S_$wx$h_scrot.png

The parts using % are the standard strftime(3) format specifiers, $w is the image width and $h is the image height.

Example:

$ scrot
$ find *.png
2020-03-07-152236_1920x1080_scrot.png

For UTC timestamps:

$ TZ=UTC scrot
$ find *.png
2020-03-07-152236_1920x1080_scrot.png
2020-03-07-183257_1920x1080_scrot.png

Note: The timezone is not specified in the filename. Considering the tool's intended purpose (i.e.: screenshots), I do not think that it would add any value. Furthermore, I find the format above easy enough both to read and to parse as it is.

Source:

if (!opt.output_file) {
  opt.output_file = gib_estrdup("%Y-%m-%d-%H%M%S_$wx$h_scrot.png");
  opt.thumb_file = gib_estrdup("%Y-%m-%d-%H%M%S_$wx$h_scrot-thumb.png");
} else {
  have_extension = scrot_have_file_extension(opt.output_file);
}

Answer 4

No separator between components makes it look like random numbers or just a huge counter.

Colon is a killer as it cannot be used on Windows systems (and thus also not on a mounted SMB share which sometimes is the only network protocol avaiable) and has unexpected side effects on macOS despite being a UNIX system: the file system is fine with colon but Finder displays it as forward slash (/) - this has historical reasons (historical colon was used as a path component separator on Mac and thus to make all paths look the same in macOS the system would map colon to slash; nice side effect, you can use use slashes in file names, as those are mapped to colons).

All dashes is confusing as well.

For time you can use underscore, period, comma or percent (all filesystems allow percent and it has no meaning in bash/sh/zsh on its own, it is only ugly in URLs as it requires escaping). None of these require escaping on shell. Or you can use semicolon which is closest to a colon but requires escaping on shell.

And the convention almost everyone follows is "If there is no timezone written, it is UTC" and generally keeping all times always in UTC makes it easier to work with them. If required, it is always possible to convert them to local timestamps.

Answer 5

I use $ date -u +%Y-%m-%d-%H-%M-%S

it's a UTC timestamp, to the best guess of my system, that encodes without hassle into:

• URLs (buckets, static hosting)
• filesystems (windows, macOS, linux)
• the shell
• other apps and GUI programs

2022-12-12-19-19-10.log
2021-11-06-15-59-00.log

• files look just readable enough.
• It's a simple character set {0,1,2,3,4,5,6,7,8,9,-}
• Does not differentiate between date and time they both are set as part of a datetime stamp
• downside, needs to be converted into iso during runtime or in database

Answer 6

1. I too wouldn't include timezones. Your scripts/tools which process the logs should know about that. Also in regard of summer/winter time changes - I'd recommend to keep your server always fixed at UTC all the time. A sudden difference between the base server timezone and an (unchanged) timezone of a database running on it may lead to headaches ;-).

2. Regarding log file naming - I know, many don't like it, but I like to keep it simple:

p-%s-type.log = p-1311116459-type.log

pros:

• common denominator
• very easy to use in further scripting

cons:

• not human readable

On machines where colleagues (for whatever reason) need to check logs manually I went for this variant, rotating daily:

p-%Y-%m-%d-type.log = p-2011-07-20-type.log

Best regards