1

I have been trying to use the AWS "official" command line tool s3curl.pl to test out the recently announced multi-object delete. Here is what I have done:

First, I tested out the s3curl.pl with a set of credentials without a hitch:

$ s3curl.pl --id=s3 -- http://testbucket-0.s3.amazonaws.com/|xmllint --format -
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100   884    0   884    0     0   4399      0 --:--:-- --:--:-- --:--:--  5703
    <?xml version="1.0" encoding="UTF-8"?>
    <ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
      <Name>testbucket-0</Name>
      <Prefix/>
      <Marker/>
      <MaxKeys>1000</MaxKeys>
      <IsTruncated>false</IsTruncated>
      <Contents>
        <Key>file_1</Key>
        <LastModified>2012-03-22T17:08:17.000Z</LastModified>
        <ETag>"ee0e521a76524034aaa5b331842a8b4e"</ETag>
        <Size>400000</Size>
        <Owner>
          <ID>e6d81ea69572270e58d3814ab674df8c8f1fd5d502669633a4951bdd5185f7f4</ID>
          <DisplayName>zackp</DisplayName>
        </Owner>
        <StorageClass>STANDARD</StorageClass>
      </Contents>
      <Contents>
        <Key>file_2</Key>
        <LastModified>2012-03-22T17:08:19.000Z</LastModified>
        <ETag>"6b32cbf8219a59690a9f69ba6ff3f590"</ETag>
        <Size>600000</Size>
        <Owner>
          <ID>e6d81ea69572270e58d3814ab674df8c8f1fd5d502669633a4951bdd5185f7f4</ID>
          <DisplayName>zackp</DisplayName>
        </Owner>
        <StorageClass>STANDARD</StorageClass>
      </Contents>
    </ListBucketResult>

Then, I following the s3curl.pl's usage instructions:

s3curl.pl --help
Usage /usr/local/bin/s3curl.pl --id friendly-name (or AWSAccessKeyId) [options] -- [curl-options] [URL]
 options:
  --key SecretAccessKey       id/key are AWSAcessKeyId and Secret (unsafe)
  --contentType text/plain    set content-type header
  --acl public-read           use a 'canned' ACL (x-amz-acl header)
  --contentMd5 content_md5    add x-amz-content-md5 header
  --put <filename>            PUT request (from the provided local file)
  --post [<filename>]         POST request (optional local file)
  --copySrc bucket/key        Copy from this source key
  --createBucket [<region>]   create-bucket with optional location constraint
  --head                      HEAD request
  --debug                     enable debug logging
 common curl options:
  -H 'x-amz-acl: public-read' another way of using canned ACLs
  -v                          verbose logging

Then, I tried the following, and always got back error. I would appreciated it very much if someone could point out where I made a mistake?

$ s3curl.pl --id=s3 --post multi_delete.xml -- http://testbucket-0.s3.amazonaws.com/?delete
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><StringToSignBytes>50 4f 53 54 0a 0a 0a 54 68 75 2c 20 30 35 20 41 70 72 20 32 30 31 32 20 30 30 3a 35 30 3a 30 38 20 2b 30 30 30 30 0a 2f 7a 65 74 74 61 72 2d 74 2f 3f 64 65 6c 65 74 65</StringToSignBytes><RequestId>707FBE0EB4A571A8</RequestId><HostId>mP3ZwlPTcRqARQZd6gU4UvBrxGBNIVa0VVe5p0rqGmq5hM65RprwcG/qcXe+pmDT</HostId><SignatureProvided>edkNGuugiSFe0ku4eGzkh8kYgHw=</SignatureProvided><StringToSign>POST


Thu, 05 Apr 2012 00:50:08 +0000

The file multi_delete.xml contains the following:

cat multi_delete.xml
<?xml version="1.0" encoding="UTF-8"?>
<Delete>
    <Quiet>true</Quiet>
    <Object>
      <Key>file_1</Key>
      <VersionId> </VersionId>>
    </Object>
    <Object>
      <Key>file_2</Key>
      <VersionId> </VersionId>
    </Object>
</Delete>

Thanks for any help!

--Zack

1 Answer 1

0

This worked for me (Amazon Linux AMI v2012.03; us-east-1):

Download and extract latest s3curl.zip

wget http://s3.amazonaws.com/doc/s3-example-code/s3-curl.zip
unzip s3-curl.zip

Make executable:

cd s3-curl
chmod +x s3-curl

Create a credential file (.s3curl):

%awsSecretAccessKeys = (
    # personal account
    personal => {
        id => 'XXXXXXXXXXXXXXXXXXXX',
        key => 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
    },
);

Restrict credentials permissions:

chmod 600 .s3curl

Create POST request body (multidelete.xml):

<?xml version="1.0" encoding="UTF-8"?>
<Delete>
    <Object>
         <Key>file1.txt</Key>
    </Object>
    <Object>
         <Key>file2.txt</Key>
    </Object>
</Delete>

Calculate base64 encoded MD5 sum of POST body:

cat multidelete.xml | openssl dgst -md5 -binary | base64
cD8Q8KTug5P8Hj7oyOW8iQ==

Construct request, and enable verbose display for curl (I have included the command above inline, instead of the MD5 sum itself):

./s3curl.pl --id=personal --post multidelete.xml --contentMd5 `cat multidelete.xml | openssl dgst -md5 -binary | base64` -- -v http://s3.amazonaws.com/BUCKET?delete

Notes:

Trying to use PUT (instead of POST) results in a 405 Method Not Allowed:

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>MethodNotAllowed</Code><Message>The specified method is not allowed against this resource.</Message><ResourceType>MULTI_OBJECT_DELETE</ResourceType><Method>PUT</Method><RequestId>XXXXXXXXXXXXXXXX</RequestId><HostId>XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</HostId></Error>

Trying to use POST without the Content-MD5 header results in a 400 Bad Request:

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidRequest</Code><Message>Missing required header for this request: Content-MD5</Message><RequestId>XXXXXXXXXXXXXXXX</RequestId><HostId>XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</HostId></Error>

Using a hexadecimal md5 sum results in a 400 Bad Request:

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidDigest</Code><Message>The Content-MD5 you specified was invalid.</Message><RequestId>XXXXXXXXXXXXXXXX</RequestId><Content-MD5>703f10f0a4ee8393fc1e3ee8c8e5bc89</Content-MD5><HostId>XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</HostId></Error>

A valid request/response looks like:

./s3curl.pl --id=personal --post multidelete.xml --contentMd5 `cat multidelete.xml | openssl dgst -md5 -binary | base64` -- -v http://s3.amazonaws.com/BUCKET?delete
* About to connect() to s3.amazonaws.com port 80 (#0)
*   Trying 72.21.194.31... connected
* Connected to s3.amazonaws.com (72.21.194.31) port 80 (#0)
> POST /BUCKET?delete HTTP/1.1
> User-Agent: curl/7.19.7 (i386-redhat-linux-gnu) libcurl/7.19.7 NSS/3.12.9.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2
> Host: s3.amazonaws.com
> Accept: */*
> Date: Thu, 05 Apr 2012 01:50:53 +0000
> Authorization: AWS XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX=
> Content-MD5: cD8Q8KTug5P8Hj7oyOW8iQ==
> Content-Length: 172
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< x-amz-id-2: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
< x-amz-request-id: XXXXXXXXXXXXXXXX
< Date: Thu, 05 Apr 2012 01:50:55 GMT
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Server: AmazonS3
<
<?xml version="1.0" encoding="UTF-8"?>
* Connection #0 to host s3.amazonaws.com left intact
* Closing connection #0
<DeleteResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Deleted><Key>file1.txt</Key></Deleted><Deleted><Key>file2.txt</Key></Deleted></DeleteResult>

Using <Quiet>true</Quiet> results in the following body returned from a successful deletion:

<DeleteResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"></DeleteResult>

The only way I could replicate your error message was by using an invalid secret key in my credentials file. However, with that the bucket listing did not work either, unlike yours.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .