Our infrastructure is on AWS. I want to get a daily report on how much spent on the previous day. What is the best way to do it?
AWS has just announced the general availability of the functionality to Monitor Estimated Charges Using Billing Alerts via Amazon CloudWatch (it apparently has been available to AWS premium accounts already since end of 2011, see Daniel Lopez' answer to Is there a way to set Amazon AWS billing limit?):
As outlined in the introductory blog post, You can start by using the billing alerts to let you know when your AWS bill will be higher than expected, see Monitor Your Estimated Charges Using Amazon CloudWatch for more details regarding this functionality.
This is already pretty useful for many basic needs, however, using the CloudWatch APIs to retrieve the stored metrics yourself (see GetMetricStatistics) actually allows you to drive arbitrary workflows and business logic based upon this data, and of course you could generate a daily report on how much spent on the previous day like so as well.
Regarding the latter, the scope of this offering is stressed as well though:
That is, the granularity of the reported metrics has yet to be analyzed (I see data points every 4 to 8 hours, but not necessarily updated values every time, as one would expect actually), so deriving a sufficiently precise daily report might require some statistical post processing.
Unfortunately this is less straight forward than one would think, especially given that the desired data can be inspected manually via your account. There are two monitoring options one would expect:
Neither AWS nor any other IaaS/PaaS/SaaS vendor I'm aware of does offer API access to their accounting data currently (maybe due to the potential financial/legal implications), making any form of 3rd party integration (which would be easy to do nowadays) cumbersome at best, insofar you need to resort to web scraping to retrieve the data in the first place.
Fortunately a new offering from Cloudability has entered the stage recently to do just this for you in a professional and vendor agnostic way, we are using it with great success already for AWS specifically - you'll currently receive a daily (or less frequent) report of your monthly spending only though, i.e. not broken down to your daily spending yet. Adding the daily increase would be trivial of course, so I'd hope and expect they'll make more information like this available over time.
The former caveat (kept for reference below) of requiring your main AWS credentials doesn't apply anymore - AWS recently introduced New IAM Features: Password Management and Access to Account Activity and Usage Reports Pages:
Cloudability has now integrated this as well, thus you don't need to hand them your main AWS credentials anymore or spent the extra effort to establish Consolidated Billing just to gain insight into your cloud spending, see How to Setup Amazon IAM (Identity Account Management) for details.
In order to access your data you'll need to hand them your main AWS credentials, because otherwise they can't scrape your account, obviously. For AWS in particular you can still avoid this by facilitating Consolidated Billing, where you consolidate payment for multiple Amazon AWS accounts [...] by designating a single paying account, which in turn has no access to your computing resources and data.
Amazon provides your current month-to-date charges here:
Towards the top of the page it indicates how current the data is. I find it tends to lag by a few hours.
This is the most accurate and up-to-date record you can get from Amazon or anybody else at this time.
If you really need a day-to-day cost report, you'll need to use the "Usage Report" tool in your AWS account. You can request a report for each service you use, on whatever time period you want, in granularity from by hour to by month. Then it downloads a CSV.
You'll need to do some post-processing on that CSV (since it's not in cost, but in usage etc), but it will provide you the data you need to have a day-to-day cost.
Take a look at Xervmon. They provide day to day spend and usage in addition analytics on historical. They are an upcoming service provider with detailed integrations with Amazon AWS planned in next 3 months.
Some screenshots from my current account is as below. http://cloudmaya.net/~develop/demo/Slide1.jpg http://cloudmaya.net/~develop/demo/Slide2.jpg
Bunch of professionals have built and it is quite neat.
I've seen companies build their own in-house tools for this - basically they scrap the AWS billing page and on their own dashboard, display the current cost, and in one example, they divide it by the days in the month that have passed, and multiple that to get the estimated total month cost.
AWS don't offer a billing API yet (I'm sure they will in future), but there are a couple of external services that can help. One is CloudVertical (disclosure: I work here), where you can get your daily, monthly, and hourly cost, broken down by service, and for multiple accounts.
The real holy grail for a service like AWS though is not just to track daily spending, but to show insights on efficiency (cost+usage = efficiency) and also highlight opportunities for savings (ie: times to use reserved or spot instances)
Maybe this Python module on Github can help you getting started: pyec2costs (for reserved or ondemand instances).