Has anyone come across a near 100% Amazon S3 RESTful API compatible object storage system?
What I am after is a layer that sit on top of any (preferably a POSIX) filesystem that provides Amazon S3 style RESTful API to store (PUT
), retrieve (GET
), stat (HEAD
) and delete (DELETE
), with decent authentication.
Commercial projects/ideas are welcome too.
NOTE:
I have so far tried Eucalyptus and Cumulus; of which Eucalyptus seems to be blindly calling itself S3-compatible. The response XML documents are not at all compatible, and is patchy in certain places with no XML documents at all. Cumulus has managed to keep the response documents quite similar, but seems to have forgotten about data integrity!
Let me explain the latter part: both Eucalyptus and Cumulus have no support for integrity verification that Amazon S3 offers. What you can do with S3 is that you can provide a Base64(MD5(FILE)) along with the PUT request, which then gets verified by S3 before they respond with a success. Eucalyptus and Cumulus doesn't support this. With, Eucalyptus we can at least get around this by checking the MD5 given in the response document (not S3-compatible behaviour). In Cumulus, this is not possible as it doesnt respond with anything (like S3). It is made even worse by Cumulus, by not giving out an ETag in the HEAD
request.