Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have been requested to investigate how I can reduce the downtime of our website upgrades.

We maintain a DNN site with both public facing pages and member only pages. The member only pages are directly linked to our core application database while the public pages are not.

Our current process is to redirect website users as soon as the upgrade process begins, which includes

  • Backup of the Prod DB
  • Update Prod DB
  • Update Executables (Application)
  • Upgrade Website Application (If this requires an update)
  • Install Dependencies
  • Upgrade sub systems like communication engine and payment broker
  • Update various configuration files
  • Perform testing of systems
  • Restart all services
  • Allow access to site

This process can take from 2 to 8 hours depending on upgrade required, scripts to be run, size of database and number or portals.

My initial thoughts are to restrict users to read only pages and any update pages would be unavailable.

Could anyone please offer suggestions as to the best practices for what I would think to be a common problem so that we can reduce this down time and if we need infrastructure changes, I can put this to our technical department.

share|improve this question

migrated from stackoverflow.com Sep 24 '12 at 2:47

This question came from our site for professional and enthusiast programmers.

    
good question. but its more fit at ServerFault. –  naveen Sep 24 '12 at 2:08
    
Whilst this is partly an administration issue, I will have to make code changes to manage the way the website needs to respond depending on suggested solutions. I'm happy to move it over to ServerFault if that's going to get a better response. –  nolan.sipos Sep 24 '12 at 2:23

2 Answers 2

Do you practice any agile deployment methodologies? My first suggestion is to measure what part of the deployment is taking longer than desired and optimize it. Try to separate database deployment from code deployment schedules.

Since you are backing up your database nightly (correct?), There shouldn't be too much data to rollback if you have to. Not a database expert, but I'm sure you can simply update the database schema where it needs to be updated with scripts without taking too much time.

You can easily deploy dlls and resources to the site via a script without interrupting existing sessions.

You should do a majority of your testing on a staging server before deploying to production. So, once you've deployed, you don't need to test as much on production.

I would encourage you to look into continuous integration or continuous deployment (http://continuousdelivery.com/).

share|improve this answer
    
We practice agile development, but not agile deployment from my understanding. You say to seperate database from code deployment, however in virtually every upgrade we would have schema updates or general data update scripts run. A specific upgrade could have scripts that require over an hour to run (not common but possible). –  nolan.sipos Sep 24 '12 at 1:51

I would suggest:

a) set site RO

b) do db backup // should be quick. if it isn't, use redgate or something to make it fast

c) db update: // should be quick. If it isn't, you are making changes that are too large too often.

d) the code should all have been pretested on staging. You should be able to bring the site up and then test it while it is live (because you have a very high confidence that the tests will pass)

We have a very large .net app, on top of a 500gb sql server db, and our steps are the same as yours, but we have only a few minutes of downtime to do an upgrade (which we do several times a week).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.