up vote 0 down vote favorite
share [g+] share [fb]

I keep on reading/misunderstanding in&out in EBS costs $.10 per million. does that mean 1M queries cost $.10? b/c i am creating a web-based tests and they require a lot of queries to save progress, results and exams. I also use blob object in my sql to store images. So does that mean i pay $.10 for 1m queries?

link|improve this question
2  
Don't store images as BLOBs. It's a horrible dirty way to do it. You'd be better off storing them as files on the filesystem, and then serving them with a decent HTTP server. – Tom O'Connor Jun 8 '11 at 13:30
Or use S3 to store them. – Raphink Jun 8 '11 at 13:33
1  
Hey tom still the op, but i have over 3000 images do you still believe i should store them as files? because when my user uploads images they are over 1mb. so what i do is use php functions ie imagecopy to make it 50kb and store it in blobs. Do you still think its dirty? but if we store in filesystem dont we have to worry about redundancies, unless if we have a naming convention and store each of those name associated with each question. etc... – nik Jun 8 '11 at 13:36
also will it be expensive to do 120 queries for each user who takes an exam? or should i just store everything in session and just do one query when they click submit or save. Wish aws/EBS is it expensive to do qeuries b/c i see costs for I/o and 128Kb do not understand what that means. could you guys please give me a quick overview thanks nik – nik Jun 8 '11 at 13:38
Yep, still dirty, regardless of anything you say to the contrary. What's so bad about having a naming convention? Look at it this way, if you store them as blobs, you end up with fecking massive SQL dumps, AND have to process the images with PHP to get them out of the database. The overhead of that alone should be significant. – Tom O'Connor Jun 8 '11 at 13:48
show 6 more comments
feedback

2 Answers

up vote 2 down vote accepted

EBS is raw, block-level storage. Think of it like virtual disk. You ask for an EBS volume, mount it on an AWS machine instance, format it, and then store data on it. It costs money to store data on an EBS volume, and money to move data in and out to an EBS share from outside AWS. Accessing it from within AWS is free, you just pay for the storage in this case.

So if you're running an SQL server on an AWS instance that stores it's data on an EBS volume, it costs you to store the SQL data on the EBS disk, but there's no cost to make a query against that SQL server running on the AWS instance (assuming the transaction can occur with data that's entirely memory mapped). That's just over-the-wire data transfer and it's what AWS instances were, more or less, designed to originally: serve up data.

To estimate your EBS costs you need to figure out how much data on disk your SQL server will to be storing, not how many queries are going to be made against your SQL server. Data on disk: that's the EBS cost of your AWS-based design. There's a section at the end of that EBS link that gives you the following advice about estimating your EBS costs:

With Amazon Elastic Block Store, you only pay for what you use. Volume storage is charged by the amount you allocate until you release it, and is priced at a rate of $0.10 per allocated GB per month Amazon EBS also charges $0.10 per 1 million I/O requests you make to your volume. Programs like IOSTAT can be used to measure the exact I/O usage of your system at any time. However, applications and operating systems often do different levels of caching, so you will likely see a lower number of I/O requests on your bill than is seen by your application unless you sync all of your I/Os to disk.

As an example, a medium sized website database might be 100 GB in size and expect to average 100 I/Os per second over the course of a month. This would translate to $10 per month in storage costs (100 GB x $0.10/month), and approximately $26 per month in request costs (~2.6 million seconds/month x 100 I/O per second * $0.10 per million I/O).

Snapshot storage is based on the amount of space your data consumes in Amazon S3. Because data is compressed before being saved to Amazon S3, and Amazon EBS does not save empty blocks, it is likely that the size of a snapshot will be considerably less than the size of your volume. For the first snapshot of a volume, Amazon EBS will save a full copy of your data to Amazon S3. However for each incremental snapshot, only the part of your Amazon EBS volume that has been changed will be saved to Amazon S3.

Volume data is broken up into chunks before being transferred to Amazon S3. While the size of the chunks could change through future optimizations, the number of PUTs required to save a particular snapshot to Amazon S3 can be estimated by dividing the size of the data that has changed since the last snapshot by 4MB. Conversely, when loading a snapshot from Amazon S3 into and Amazon EBS volume, the number of GET requests needed to fully load the volume can be estimated by dividing the full size of the snapshot by 4MB. You will also be charged for GETs and PUTs at normal Amazon S3 rates.

link|improve this answer
Thank you very much. – nik Jun 8 '11 at 13:50
@nik: I should add "data on disk and queries against that disk" to my last paragraph since you are charged for I/O requests to the EBS volume. And the SQL server can make many queries against disk to bring data in to memory to serve up front end query requests. – Ian C. Jun 8 '11 at 13:52
feedback
up vote 0 down vote

That $0.10 is charged based on number of I/O requests not number of queries. If you use EBS to store a database, then each database query may require any number of I/O requests. For instance, an UPDATE query takes several operations to complete: it needs to read through the database to find the rows to change, then write back the changed rows and possibly write changed indexes as well. However, if the database server has cached the table in memory, it will not need to read from the EBS volume to find the rows to change, so it will take fewer I/O requests. There isn't an easy way to say "$0.10 buys you this many queries".

link|improve this answer
ohh ok. Do you think for my use it would be better to just store the users answers in a php session and then update it when the user finishes his exam. or should i be constantly updating for each question. thank you very much for your help. – nik Jun 8 '11 at 13:55
@nik: that kind of software design decision might be better asked over at stackoverflow.com. – Ian C. Jun 8 '11 at 14:15
@nik: You should be thinking about more than "how much is this going to cost me". You also need to think about how your design impacts the users. What should you do with half-done exam data, if someone quits mid-test? If the user doesn't hit the ending page that saves the session to the database, none of their answers will be there unless you handle that outside of the web application somehow. What if you need to start a new EC2 instance and the session data for half-done tests isn't on the EBS volume? – DerfK Jun 8 '11 at 16:42
Hey derfk, i have though about these things are you are absolutely correct. Its better to play. Thank you for your ideas. – nik Jun 8 '11 at 20:38
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.