Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am trying to keep long-term logs of an app in such a way, that it could plausibly demonstrated to third parties/court that the application has processed certain data at a given time. The data can be represented in XML or text format. A simple gzipped log is not plausible, as I may have added or modified data afterwards, whereas an external logging service would be an overkill. Cost is an issue, we are not dealing with financial data or so, but rather some simple user generated content, where some malicious users tried to blame the operator in the past when things escalated and went to court.

My question:

  1. Is there some kind of signing software for Linux that signs each element of a log in such a way, that it can be easily shown that no element can be added or modified afterwards? Plug-Ins into some free Splunk Alternatives would be fine too.

  2. Ideally the software I am looking for should be under a GPL or similar license. I could probably achive something like this by using PGP/GPG sgning functions and including the previous elements signituares within the following element, but I would prefer to use some program where you do not have to argue about the validity of your own code.

Note to mods: I am not asking this question on Stackoverflow, because I am not looking for writing own code for reasons described above. I think this question rather fits into serverfault than superuser, as server-side logging software is discussed rather here than on superuser.

share|improve this question

closed as off topic by John Gardeniers, HopelessN00b, Tom O'Connor, Ward, Michael Hampton Oct 7 '12 at 13:32

Questions on Server Fault are expected to relate to server, networking, or related infrastructure administration within the scope defined by the community. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about reopening questions here.If this question can be reworded to fit the rules in the help center, please edit the question.

While you logic in choosing the most appropriate site to post to initially seems sound, shopping questions are off topic on all Stack Exchange sites. – John Gardeniers Sep 29 '12 at 21:41
@JohnGardeniers Sorry, but I do not consider it a "shopping question". I have a server related task, namely to make logging somehow verifiable by third parties without going a full compliance solution way. I tried to explain in my question, why a complete home-made solution would not be helpful. I tried to keep the question as open as possible, if I had asked, how can I make Splunk compliant you would have probably not considered it a "shopping question", although it has exactly the same result. – Martin Sep 30 '12 at 0:36
Please read our FAQ. You are asking for a product recommendation, not a procedure, which not only makes it a shopping question but is explicitly declared off topic. – John Gardeniers Sep 30 '12 at 1:56
You can't do anything for free, certainly not in a manner that would satisfy third parties. Fork out, or feck off. – Tom O'Connor Sep 30 '12 at 9:21
up vote 1 down vote accepted

Consider trusted timestamps. It might be enough to use openssl ts to sign the logfile hashes.

share|improve this answer

I'm going to recommend you take a look at Amazon Glacier. It's 1 cent per month, per gigabyte that you want to store. So unless your logs are massive, there shouldn't be a massive cost problem.

The main reason I'm telling you this is because, from their FAQ:

Q: Can I see what archives I have stored in Amazon Glacier? Yes. Although you will need to maintain your own index of data you upload to Amazon Glacier, an inventory of all archives in each of your vaults is maintained for disaster recovery or occasional reconciliation purposes. The vault inventory is updated approximately once a day. You can request a vault inventory as either a JSON or CSV file and will contain details about the archives within your vault including the size, creation date and the archive description (if you provided one during upload). The inventory will represent the state of the vault at the time of the most recent inventory update.

The main idea is that there's proof in the form of Amazon's records that you uploaded the records when you said you did, and they have held them since then.

(emphasis mine).

share|improve this answer
Thank you very much, this is an option. I guess I will include an md5 of the file as description before uploading it into the archive, it does not seem to provide this information itself, which would have been nice for this case. Anyway I will test it and see myself. – Martin Sep 29 '12 at 18:22
You could have changed the description after you uploaded it (assuming they offer this). However, API offers modification dates as well so you can/should be able to easily justify the vault hasn't been changed since it was uploaded, and thus the logs have not been tampered with. – Jay Sep 29 '12 at 18:24
For the truly paranoid, compress, encrypt, hash/sign, and THEN upload the files, also with the hash/signature pair. Automate it all nicely and your malicious lusers will have trouble dodging a perjury or defamation charge if they pull that shit again. – HopelessN00b Sep 30 '12 at 2:32
At no point, did I even suggest it was a universal solution to storage problems. This user doesn't have 20TB of logs, or it wouldn't be cost effective, and they wouldn't consider it. Further, it's incredibly unlikely this user would want to retrieve all their logs at the same time, since they'd only need the logs tied to a particular user or case. On that basis, I felt it was a good option. – Jay Sep 30 '12 at 9:32
I can only agree with the comment by @Jay regarding my usage case. The Glacier suggestion was in my case almost perfect, the costs involved neglegible, so I will probably employ the solution as well as the ts solution proposed by mschuett. – Martin Oct 2 '12 at 22:02

Not the answer you're looking for? Browse other questions tagged or ask your own question.