I'm dealing with some data that's governed by specific regulations and that must be handled in a specific manner.

I'm finding that this data ends up in some of my log files as a result of the system operating as intended. I'd like to find a way to log messages on the server that receives that data, but to do so in such a way that the data is encrypted as it's written to disk and may not be decrypted by that same server.

My thought is that there should be a filesystem (written as a FUSE system or something - I'm using Linux) that exposes the directory structure as clear text, but writes the contents of the files according to one half of an asymmetric key set. This would permit me to log the messages and queue them to be sent off to a log server where the decryption key resides.

The inefficient (CPU-wise) nature of asymmetric encryption may make this infeasible, but I suspect that there may be a solution out there. I haven't found anything in my searching yet; is there a solution out there that can operate in the described fashion? Thanks for any tips!


I'd recommend you do it in the application layer.

In that case, then move the thread to StackOverflow, however, my answer would be approximately:

  • The normal way of using asymmetric encryption on significant amounts of data (i.e. anywhere) is to use it to encrypt a randomly chosen key which is then used to encrypt the rest of the data using a symmetric cipher
  • This is because symmetric ciphers are much faster and generally do more of what you want

I'd design it such that the log file was written in chunks which contained a header containing a key encrypted using an asymmetric cipher, which would then be used to encrypt a certain amount of data using a symmetric cipher.

So for example you could use AES / CBC for the log data itself, which is going to be pretty fast (plus there are loads of implementions).

Then every so often, your app discards the key and starts a new "chunk" - once the key is gone out of ram, it cannot then decrypt the previous chunk any more.

So you might use (say) RSA to encrypt (using a public key) the symmetric key for a chunk, which would be securely randomly chosen for that chunk. The symmetric key then stays in memory for a while, until the chunk ends, and it is wiped out.

A filesystem which supported this would be of limited usefuless because you could only mount it write-only or read-only; operating systems don't normally understand the concept of write-only filesystems :)

| improve this answer | |
  • Thanks! Your last paragraph is spot on; I was hoping that there might be a solution that allows an application to write the contents of a file (append, only) while only exposing the file statistics (name, size, permissions, etc.) to applications and the operating system. My problem is dealing with Apache logs, specifically. I may just try to find a way to keep those logs from being written locally at all and have them only shuttled off to another system. I realized after I wrote the above question that that would be a much simpler solution. :) – Justin Thomas Nov 30 '09 at 3:56

I did some research, coding and documentation; here's a link to a usage scenario...


...I wrote just for you and server admins facing similar regulations. Test it before putting into production and read the warnings.

I know that links alone aren't sufficient to answer the question so I'll go into the thought process behind the proposed solution.

Because logs are written to a few lines at a time making a one way encrypted file system is a bit over kill and outside the skills I've got, instead the script automates setting up a specific file type (mkfifo) that acts as a named-pipe, and a set of scripted rules that listen for writing actions on the named pipe file. This pair of files then generates the third file, the one you've been asking for, an encrypted appended log file that the host can't read. The script's operations are similar to echo 'log message' | gpg -a -r email@host.domain >> server.log running for every log action but with a named pipe instead of |and the script's logic handles the gpg'ing and appending to a file. Using named pipes instead of anonymous pipes means that only the log generating service's output path needs updating to use the named pipe's path instead it's standard file to make use of encrypting tasks.

Note there are other features that maybe useful already baked in that allow for encrypting and emailing log files when they reach a user specified size, these features are separate to allow IDS/IPS software a chance to parse logs or to make shipped logs even harder to snoop when combined with per-write encryption.

Edit/update: ya may want to check the abbreviated quick start for another question that was more general in scope but similar enough to warrant inclusion within this answer.


Edit/Update: looks like links within markdown on GitHub is very browser specific, so here's a copy of scenario one from the above linked document.

Custom usage scenario for web server

  • Download and copy script to a ${PATH} dir


git clone https://github.com/S0AndS0/Perinoid_Pipes
cd Perinoid_Pipes
cp Paranoid_Pipes.sh /usr/local/sbin/
chown ${USER}:${USER} /usr/local/sbin/Paranoid_Pipes.sh
chmod 700 /usr/local/sbin/Paranoid_Pipes.sh


  • Show available command line options & their current values.


Paranoid_Pipes.sh --help


  • Set command line options to write a named pipe within the same file system as the web server app and custom scripted parser on the hosting file system.


Paranoid_Pipes.sh --copy-save-yn='yes'\
 --disown-yn='yes' --help


Note if the server app is not within a chroot jail you'll want to modify file paths for the named pipe and script copy because above assumed some form of file system segregation has already been implemented. Remove the --help from above to have the script run actions instead of printing command line options' current values.

The named pipe file path (the --named-pipe-name file path) should be manually added to your web server's logging path, this is different for every daemon/server, such that your normal clients logged data is written to the named pipe file.

The script copy (saved to --copy-save-name file path) is what does the magic by listening for write actions to its related named pipe file, it pushes logged lines through GnuPG and appends the results (using the public key for --output-parse-recipient for initial encryption) to the file specified by the --output-parse-name command line option.

The resulting encrypted log file is monitored for file size and when that and the write counter are large enough the log file is re-encrypted using the public key for --output-rotate-recipient and moved/renamed.

If your server is able to send email then modify the --output-rotate-actions options to include email as a valid action and the doubly encrypted logs will eventually be transferred.

Note if you use two different public keys for encryption then you will need the related private keys to be applied in the correct order for decryption.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.