2

The doc says

If you enable versioning for a bucket, Amazon S3 automatically generates a unique version ID for the object being stored

When versioning is enabled for a bucket, can I say version id returned is unique for all the files in the bucket ?

I could not get enough details from their docs. Have I missed anything ?

My concern is Amazon says that

There is no limit to the number of objects that can be stored in a bucket and no difference in performance whether you use many buckets or just a few

But the version id's length is 32.

Length of version id is 32

So it can generate up to 2^32 id's i.e 4,29,49,67,296, right ? So is that a limitation on number of objects that can be created in a bucket ?

Thanks in advance.

2
  • 1
    Looks more like 32 characters, not 32 bits. That's very close to "unlimited".
    – Oliver
    Dec 1, 2014 at 11:32
  • I'd agree with that. Last time I did something similar, I certainly had no problems. I have to say though, if your intending to store so much that you'd go over the 32 bit limits, then I'd first question what I'm trying to do and why, as it seems very excessive!
    – shawty
    Dec 1, 2014 at 11:40

1 Answer 1

4

Technically speaking, it appears they might be longer, in the future:

Unique version IDs are randomly generated, Unicode, UTF-8 encoded, URL-ready, opaque strings that are at most 1024 bytes long

http://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectVersioning.html

However, there doesn't seem to be an explicit statement as to the scope of these identifiers.

The bucket versioning documentation (now) includes this statement:

When you enable versioning for a bucket, all objects added to it will have a unique version ID.

Assuming this is correct as written, the version ID is scoped to the bucket... but I am still skeptical. S3 sometimes performs index partition splits on buckets for performance reasons, and unique version ID maintenance across partitions seems somewhat unlikely. It really depends on what the information in a version ID actually means, if it means anything at all. It's opaque, so for all we know, it could be random.

In the strictest sense from an application's perspective, they are only unique to the individual object, because if objects are created in an unversioned bucket, and versioning is later enabled on the bucket, every object that existed when versioning was enabled has the same version id, which is the literal string null when you are interacting with the REST API.

In any event, though, a version ID isn't meaningful in the absence of the acconpanying key, because you can't fetch an object by version id alone -- only by key + version id.

Taken together, the appropriate course of action would seem to be to work with the assumption that version ids are scoped to, and only meaningful at, the object key level.

3
  • Randomly generated is globally unique without the need for coordination. The fact that those are longer than 128 bit suggest they might. Even contain some more info (like the object Id or something) Bit the guarantees in the documentation look more like one should store them always relative the the actual object name.
    – eckes
    Apr 27, 2017 at 1:15
  • Randomly generated is globally unique without the need for coordination. No, it isn't, @eckes. Not perfectly so. Regardless of the quality of a random number generator, collisions are not impossible since the odds of generating a given value are no lower than those of generating any other value... generated values still have to be checked for duplication if there is a guarantee of absolute uniqueness. In current practice I see just 32 characters of modified base-64 = only 192 bits long, since base64, having only 2^6 (64) symbols, encodes only 6 bits per character, instead of 8. Apr 27, 2017 at 1:53
  • Well, the world would burn down if 128 uniform distributed bits (UUID type 4) collide. Of course nobody stops you from testing.
    – eckes
    Apr 27, 2017 at 1:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .