I am currently editing my Python code (Flask Web Server) on my live production server. I'm just getting a website started and understand this is not a viable long-term solution. I understand that many people have many different approaches for managing a development, a testing, and a production environment (and probably more), but can someone please point me to a place where can I can get started learning more about managing this type of process flow? I don't know where to begin. Thanks.
3 Answers
Let's stop for a minute and recall why we separate development and production environments in the first place.
In short, it's because we are human and make mistakes. The point of having separate environments is to catch and correct mistakes as early as possible, and before they impact our users.
There are almost as many ways to set this up as there are applications to be deployed. Whatever you are doing, your best bet is to use a methodology which is simple enough to understand and use that you aren't tempted to work around it, and correct enough to help you catch problems before they reach production.
Now if you want to be enterprisey about it, and set up something that almost nobody understands in its entirety and almost nobody actually likes, then by all means follow ITIL. There's enough process in there to keep you buried in paperwork for half your career. Though, you may have to do this if you are developing software in certain industries or for certain clients.
From your question, though, it sounds like you are a single developer or a small team, working on a project which needs a bit more flexibility and speed.
In essence you need two types of tools: testing tools and deployment tools.
At minimum, all you really need is two servers: the production server, and a testing (staging) server. You'll first test your changes on your local workstation. If everything seems all right, you push your code to the staging server, and test it again. If everything still seems OK, you push it to production and wait for the fire alarm to go off.
One of the most powerful methodologies for testing your software is continuous integration. With this method, whenever you check in code to a shared repository, a server checks out the code and runs your entire test suite. (You did write unit and functional tests for everything, right?) It can also do integration testing. If all the tests pass, you can configure such tools to promote the code to production and deploy it automatically. Github does this, for instance. Two common CI tools today are Jenkins and Travis.
Finally there's deployment. This seems obvious: you copy your new code to the (staging or production) server and then tell the web server to start new workers on the new code. Tools such as Capistrano, Commando or Deployer help automate this. They can also typically roll back to a previous deployment if something goes wrong. (And your language or framework should have ways to deal with rolling back database schema changes, such as migrations in Rails. For Flask, you might use something like SQLAlchemy and Alembic.)
The end state you probably want to be in is one where you have 100% test coverage, with unit, functional and integration tests for everything, so that you can take advantage of continuous integration to deploy your changes -- or catch mistakes -- as rapidly as possible. Start small, with a single "testing" server, and work your way up, until you get there.
Why you shouldn't run development in production
Anyone can run arbitrary code on your server via debugging utilities available (that's if you're in debug mode, which you would be if you were developing your site.)
How you can set up your environment
- Set up a simple NGINX reverse proxy.
- Setup a
staging.yourwebsite.com
site with its own folder - Set up a production site with its own folder
- Limit the staging site to your home IP address only
An easy NGINX config example:
upstream production {
server 127.0.0.1:5000; # Default Flask address
}
upstream staging {
server 127.0.0.1:5001;
}
server {
server_name yourwebsite.com default_server;
access_log /var/log/nginx/yourwebsite_access.log;
error_log /var/log/nginx/yourwebsite_error.log;
location / {
proxy_pass http://app;
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
server {
server_name staging.yourwebsite.com;
access_log /var/log/nginx/staging_access.log;
error_log /var/log/nginx/staging_error.log;
allow YOUR_IP_ADDRESS;
deny all;
location / {
proxy_pass http://staging;
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
I really don't understand your question/problem.
However there is a methodology to put something from Developpement to Production : it is called ITIL
This is "all text" and provides/describes best pratice and process to put anything from Dev to Prod, minimizing and controlling risks...
-
4
-
Yeah for sure ;) But i cannot see what else to say...To me the workflow he is looking for is this...– krisFRAug 7, 2014 at 22:37
-
I'm just looking for guidance on how to run a reasonably controlled service. Once I go live and have to make updates, is there any mechanism out there that facilitates or assists with this step? If not, I assume I would have to shutdown server, update code, update database if it needs a new table, and then restart service. I just thought there would be a more robust way of doing this.– RyanAug 7, 2014 at 22:53
-
3What are you talking about ?? your original question has nothing to do with some database updates and restarting services. Please be clear– krisFRAug 7, 2014 at 22:57