0

I'm developer and have 0 sys admin knowledge.

Currently, I had built several apps on the top of Google App Engine. I'm pretty happy with it, as it requires 0 sys admin effort, to achieve such scale.

In term of storage, I'm using

  • High Replication Datastore
  • Cloud SQL with backups enabled (Binary log disabled)

I know Google infrastructure is pretty reliable. However, for paranoid reason, I was wondering, is there any need for me to export both Datastore & Cloud SQL data, and store in either in my local machine, or non Google server?

Or, my current app settings (high replication on datastore, and backups enabled in Cloud SQL) are good enough for me to be in peace of mind, and requires 0 action from my side?

p/s Cross post at https://groups.google.com/forum/#!topic/google-appengine/ZEfV21Fbdkc

1
  • 1
    The general problem with replication is that it is typically not a back-up, because erroneous, malicious and accidental delete, modify and overwrite actions get replicated too and are typically not recoverable. (I am unfamiliar with that particular data store, because potentially a data store can be designed such that both modify and overwrite generate a new versions and nothing actually gets deleted or overwritten (just labelled as deleted/superseded) and you can recover a prior state)
    – HBruijn
    Jul 19, 2015 at 6:28

3 Answers 3

2

is there any need for me to export both Datastore & Cloud SQL data, and store in either in my local machine, or non Google server? - Do you trust Google with your data? Does Google guarantee that they won't lose your data?

Or, my current app settings (high replication on datastore, and backups enabled in Cloud SQL) are good enough for me to be in peace of mind, and requires 0 action from my side? - We can't answer that. Do you have peace of mind with the current solution? My guess is no, otherwise you wouldn't be asking. We can't answer the question as to what will give you peace of mind. If it were me, I would make sure I had an offsite backup of my data if that data is important to me, regardless of what the provider/vendor says or offers.

1

If the data size is fairly manageable, I periodically download backups either locally or to cloud storage. This allows me to bring that data into a dev/test environment to help optimize performance of some queries using real data.

That copy of the data may also help if one of my clients accidentally deletes a lot of their data. I can use the backup from a prior date to compare data.

1

Although it is still in Beta (and thus not yet recommended for production deployment), Google Admin Console does offer handy ways to backup your datastore.

It comes with many caveats and I can't repeat all of them here, so I strongly recommend you read the page at that URL; essentially, you need to "Enable Datastore Admin" in the Admin Console (you need do it but once), then (each time you want a backup) set up the details of the backup you want, and execute it (there are also "scheduled backups", but they're unfortunately still in Alpha, so, not recommended for anything but experimentation).

The best destination for Datastore Admin backups is a Google Cloud Storage bucket; if you're really paranoid, you may also want to backup that bucket to your local disks, or to S3, e.g with the gsutil rsync command.

Like you, I'm more a developer than a system administrator; however, I live by the wise words of a sysadm friend -- "nobody ever tore out their hair in desperation because of having backed things up too much"... while the reverse (hair torn out due to not having backed things up enough) is far from a rare phenomenon... so, erring on the side of caution seems good.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .