Django Full Stack Set Up and Deployment
28th February, 2017
After installing all the basic set up for Django web deveopment ( http://www.shubhamdipt.com/blog/mac-set-up-for-django-web-development )
Step by Step process: (for Django, PostgreSQL)
[ replace <project> by your project's name]
- Create virtual environment
mkvirtualenv <project>
- Activate the environment
workon <project>
- Install Django
pip install django
- Create a Django project
django-admin startproject <project>
- Enter the project diretory
cd <project>
- Initiate Git (version control)
git init
- Add remote repository to git
git add remote <repo name>
- Create .gitgnore file
vim .gitignore
- Create directories: static, templates, locale, requirements
- Inside requirements folder, create base.txt, local.txt, production.txt files.
- Create a database in postgresql <project_name> using pgAdmin.
- Install psycopg2 for postgresql.
pip install psycopg2
- Install whitenoise to serve static files in production.
pip install whitenoise
- Install raven for Sentry logging.
pip instal raven
- Install cloudinary
pip install cloudinary
- Save the requirements in requirements.txt file.
pip freeze > requirements.txt
- Copy the follwing to the settings.py
and replace <project> with your project's name.""" Django settings for <project> project. Generated by 'django-admin startproject' using Django 1.10.5. For more information on this file, see https://docs.djangoproject.com/en/1.10/topics/settings/ For the full list of settings and their values, see https://docs.djangoproject.com/en/1.10/ref/settings/ """ import os import json import cloudinary import raven PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) DEP_NAME = os.getenv('DEP_NAME', 'x/local').split("/")[1] VERSION = 1 # for Redirection PROJECT_DOMAIN = os.environ.get('PROJECT_DOMAIN') # only Production environment SUBDOMAIN_REDIRECT = True SSL_REDIRECT = False '''######################################################################## ####### Django Basic ##### ########################################################################''' # Application definition INSTALLED_APPS = ( 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.contrib.sites', 'django.contrib.sitemaps', 'django.contrib.admin', 'django.contrib.humanize', ) MIDDLEWARE_CLASSES = ( 'django.middleware.gzip.GZipMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.auth.middleware.SessionAuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'whitenoise.middleware.WhiteNoiseMiddleware', ) ROOT_URLCONF = '<projet>.urls' WSGI_APPLICATION = '<project>.wsgi.application' LANGUAGES = ( # ('de', 'Deutsch'), ('en', 'English'), ) LOCALE_PATHS = ( os.path.join(PROJECT_ROOT, 'locale'), ) LANGUAGE_CODE = 'en' TIME_ZONE = 'Europe/Berlin' USE_I18N = True USE_L10N = True USE_TZ = True SITE_ID = 1 '''######################################################################## ####### HEROKU DEPLOYMENT ##### ########################################################################''' config = { key: value for key, value in os.environ.items() } aws_database_url = config.get('RDS_DB_NAME', None) heroku_database_url = config.get('DATABASE_URL', None) docker = config.get('DOCKER', None) if aws_database_url: # aws production settings DEBUG = False SECRET_KEY = config.get('SECRET_KEY') ALLOWED_HOSTS = ['*'] # For sentry settings INSTALLED_APPS += ( 'raven.contrib.django.raven_compat', ) RAVEN_CONFIG = { 'dsn': config.get('SENTRY_DSN', ''), } DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql', 'NAME': os.environ['RDS_DB_NAME'], 'USER': os.environ['RDS_USERNAME'], 'PASSWORD': os.environ['RDS_PASSWORD'], 'HOST': os.environ['RDS_HOSTNAME'], 'PORT': os.environ['RDS_PORT'], } } elif heroku_database_url: # heroku production settings DEBUG = False SECRET_KEY = config.get('SECRET_KEY') ALLOWED_HOSTS = ['*'] import dj_database_url DATABASES = { 'default': dj_database_url.parse(heroku_database_url), } # For sentry settings INSTALLED_APPS += ( 'raven.contrib.django.raven_compat', ) RAVEN_CONFIG = { 'dsn': config.get('SENTRY_DSN', ''), } elif docker: # Docker settings DEBUG = True cred_file = open(os.path.join(PROJECT_ROOT, 'local_config.json')) creds = json.load(cred_file) config = creds.get('CONFIG') SECRET_KEY = config.get("SECRET_KEY") ALLOWED_HOSTS = ['0.0.0.0'] DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql', 'NAME': 'postgres', 'USER': 'postgres', 'HOST': 'db', 'PORT': 5432, } } else: # development/test settings DEBUG = True cred_file = open(os.path.join(PROJECT_ROOT, 'local_config.json')) creds = json.load(cred_file) config = creds.get('CONFIG') SECRET_KEY = config.get("SECRET_KEY") ALLOWED_HOSTS = [] DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql', 'NAME': '<project>', } } '''######################################################################## ####### Templating ##### ########################################################################''' TEMPLATES = [ { 'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [ '{0}/templates/'.format(PROJECT_ROOT), ], 'APP_DIRS': True, 'OPTIONS': { 'context_processors': [ 'django.contrib.auth.context_processors.auth', 'django.template.context_processors.debug', 'django.template.context_processors.i18n', 'django.template.context_processors.media', 'django.template.context_processors.static', 'django.template.context_processors.tz', 'django.contrib.messages.context_processors.messages', 'django.template.context_processors.request', ], }, }, ] '''######################################################################## ####### File Handling + Storage ##### ########################################################################''' STATICFILES_DIRS = ( os.path.join(PROJECT_ROOT, 'static'), ) STATIC_URL = '/static/' STATIC_ROOT = os.path.join(PROJECT_ROOT, 'staticfiles') '''######################################################################## ####### Caching ##### ########################################################################''' # ToDo: Add Redis settings open_redis_url = config.get('REDIS_URL') if open_redis_url: CACHES = { 'default': { 'BACKEND': 'django.core.cache.backends.dummy.DummyCache', } } else: CACHES = { "default": { "BACKEND": "django_redis.cache.RedisCache", "LOCATION": open_redis_url, "OPTIONS": { "CLIENT_CLASS": "django_redis.client.DefaultClient", "IGNORE_EXCEPTIONS": True, }, 'KEY_PREFIX': DEP_NAME, 'VERSION': VERSION, 'TIMEOUT': 86400, # 1 day }, } CACHING_MAX_AGE_BASIC = 60 * 60 * 24 SESSION_ENGINE = "django.contrib.sessions.backends.db" # Cacheable files and compression support using whitenoise STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage' '''######################################################################## ####### APIs + Other Tools ##### ########################################################################''' SUIT_CONFIG = { 'ADMIN_NAME': 'Dr. Shubham Dipt', 'MENU_EXCLUDE': ('auth.group', 'auth', 'site'), 'MENU_OPEN_FIRST_CHILD': True } CRISPY_TEMPLATE_PACK = 'bootstrap3' '''######################################################################## ####### Debug Toolbar settings ##### ########################################################################''' # Debug toolbar setting if DEBUG: INTERNAL_IPS = ('127.0.0.1',) INSTALLED_APPS += ( 'debug_toolbar', ) MIDDLEWARE_CLASSES += ( 'debug_toolbar.middleware.DebugToolbarMiddleware', ) '''######################################################################## ####### Keys and Other Information ##### ########################################################################''' # Sendwithus settings EMAIL_TEMPLATE_LANGUAGES = { 'de': 'de-DE', 'en': 'en-US', } SENDWITHUS_API_KEY = config.get("SENDWITHUS_API_KEY", '') # Cloudinary settings. os.environ.setdefault("CLOUDINARY_URL", config.get('CLOUDINARY_URL', '')) cloudinary.config(secure=True) # use cloudinary always with SSL
- (optional for Production) Add middleware.py to the same folder as settings.py for specific redirects like HTTP to HTTPS and subdomain redirects.
Add PROJECT_DOMAIN="<your domain>" as environment in production server.
Add '<project>.middleware.SubdomainRedirect' to the top of the MIDDLEWARE_CLASSES in settings.py.
# -*- coding: utf-8 -*- from django.http import HttpResponsePermanentRedirect from django.conf import settings class SSLRedirect: """Redirect the URL to SSL if option enabled.""" def process_request(self, request): """Process request.""" if not request.is_secure() and settings.SSL_REDIRECT: if settings.DEBUG and request.method == 'POST': raise RuntimeError( ("Django can't perform a SSL redirect while maintaining " "POST data. Please structure your views so that " "redirects only occur during GETs.""") ) return HttpResponsePermanentRedirect( u"https://{}{}".format( request.get_host(), request.get_full_path(), ) ) class SubdomainRedirect: """Redirect the URL if the HOST is invalid.""" def process_request(self, request): """Process request.""" if settings.SUBDOMAIN_REDIRECT and settings.PROJECT_DOMAIN: if request.get_host() != settings.PROJECT_DOMAIN: if settings.SSL_REDIRECT: redirect_url = u"https://{}{}" else: redirect_url = u"http://{}{}" return HttpResponsePermanentRedirect( redirect_url.format( settings.PROJECT_DOMAIN, request.get_full_path(), ) )
- (optional) Add context_processors.py in the same folder as settings.py to determine the environment of the project. It can be useful in certain cases, for e.g., to prevent google analytics from execution or in development servers. Moreover it also serves to return the full url of the webpage.
Add '<project>.context_processors.global_settings' to the context processors in the TEMPLATES section in settings.py file.
from django.conf import settings def global_settings(request): # return any necessary values if settings.SSL_REDIRECT: complete_url = u"https://{}{}" else: complete_url = u"http://{}{}" complete_url = complete_url.format(settings.PROJECT_DOMAIN, request.get_full_path()) return { 'DEP_NAME': settings.DEP_NAME, 'WHOLE_URL': complete_url, }
- (Good practice to keep settings as config file) Add a file called local_config.json in project's folder.
{ "CONFIG": { "SENDWITHUS_API_KEY": "XXX", "SECRET_KEY": "XXX", "CLOUDINARY_URL": "XXX" } }
Note: Keep private keys in this file for running the application locally and as environment variables in production environment.
- FRONTEND set up using Node.js, Grunt, Pug
- Add the static files in static folder.
- Add the templates in templates folder.
- Add package.json, bower.json and Gruntfile.js
- PRODUCTION Set up
- Requirements settings: create a folder called requirements and inside that create three files
base.txt - which contains the basic libraries to be installed
local.txt - which extends base.txt and contains the libraries specific for testing in local environment as follows:
-r base.txt
model_mommy==1.2.5
ecdsa==0.13
Fabric3==1.12.post1
paramiko==1.17.2
pycrypto==2.6.1
production.txt - which extends base.txt as follows:
-r base.txt
dj-database-url==0.4.1
gunicorn==19.6.0
- Add gunicorn_config.py for gunicorn configuration.
import os import multiprocessing size = int(os.getenv("SIZE", 1)) # Maximum 5 worker per 128MB workers = min(multiprocessing.cpu_count() * 2 + 1, size * 1) timeout = int(os.getenv("GUNICORN_TIMEOUT", 120))
- Deployment
- Using HEROKU
- Add Procfile
web: gunicorn <project>.wsgi:application --config gunicorn_config.py
- Add runtime.txt
python-3.5.2
- Add fabfile.py for easy deployment using Fabric (note: fabric should be included in local.txt in requirements folder) as follows:
# -*- coding: utf-8 -*- import os import json import random import requests from fabric.colors import green, yellow, red, cyan, white, blue from fabric.contrib.console import confirm from fabric.api import runs_once, lcd, local, task APP_NAME = '<project>' DJANGO_SETTINGS_MODULE = '%s.settings' % APP_NAME SLACK_URL = None installations = { 'master': 'master', # PRODUCTION 'staging': 'staging', # STAGING (things REALLY should work here) 'development': 'development', # DEV (things get merged here) } heroku_app_base_url = 'https://%s-%s.herokuapp.com' urls = { 'master': heroku_app_base_url % (APP_NAME, 'master'), 'staging': heroku_app_base_url % (APP_NAME, 'staging'), 'development': heroku_app_base_url % (APP_NAME, 'development'), } '''######################################################################## ####### General FABRIC tasks ##### ########################################################################''' @task def heroku_login(): """ For Heroku logging in.""" local('heroku login') @task def push_and_deploy(deployment=None, force_push=False): """Push code to Git and then push to Heroku for deployment.""" # Getting the current git branch all_git_branch = local("git branch", capture=True).split('\n') current_branch = None for br in all_git_branch: if '*' in br: current_branch = br[2:] print(yellow("Current branch: {}".format(current_branch))) if not current_branch: print(red("No git branch initiated")) return # Verifying current branch and deployment. if deployment: if current_branch != deployment: print(red("Branch mismatch: deploying wrong branch to {}-{}".format( APP_NAME, deployment))) return else: deployment = current_branch # Messaging bar_length = 20 + len(deployment) print(green(" ")) print(green("=" * bar_length)) print(white('Pushing to Heroku: {}'.format(deployment))) print(green("=" * bar_length)) app_name_branch = '%s-%s' % (APP_NAME, deployment) heroku_login() local_push_cmd = 'git push origin {}'.format(deployment) add_remote = 'heroku git:remote -a {}'.format(app_name_branch) push_cmd = 'git push heroku {}:master'.format(deployment) if force_push: local_push_cmd = "%s -f" % local_push_cmd push_cmd = "%s -f" % push_cmd print(green('Pushing to Git .....')) local(local_push_cmd) print(green('Adding remote Heroku app .....')) local(add_remote) print(green('Pushing to Heroku .. ')) print(yellow('Initiating a deployment to APP: {}'.format(app_name_branch))) local(push_cmd)
- Good practice: create heroku apps as <app_name>-master or <app_name>-development.
APP_NAME should be defined in fabfile.py.
Run in terminal:fab push_and_deploy
(note: this command deploys the <current-branch> to app_name-<current-branch> only.) - TIPS:
heroku run 'bash' --app <project>-master (to run bash in heroku)- python manage.py createsuperuser (to create superuser in heroku app)
- Add Procfile
- Using AWS Elastic Beanstalk
- Install awsebcli
pip install awsebcli
eb init
(and enter details of the aws environment)- Create a folder in the project folder called .ebextensions and add the following the files:
- 01_packages.config
packages: yum: git: [] postgresql95-devel: [] libjpeg-turbo-devel: []
- 02_python.config
option_settings: "aws:elasticbeanstalk:application:environment": DJANGO_SETTINGS_MODULE: "<project>.settings" "PYTHONPATH": "/opt/python/current/app:$PYTHONPATH" "aws:elasticbeanstalk:container:python": WSGIPath: <project>/wsgi.py NumProcesses: 3 NumThreads: 20 "aws:elasticbeanstalk:container:python:staticfiles": "/static/": "staticfiles/" container_commands: 01_migrate: command: "source /opt/python/run/venv/bin/activate && python manage.py migrate --noinput" leader_only: true 02_collectstatic: command: "source /opt/python/run/venv/bin/activate && python manage.py collectstatic --noinput"
- 01_packages.config
eb create
(to create the environment in aws)eb deploy
(to deploy in the created environment)- TIPS:
sudo eb ssh <aws app name>
(for ssh in the aws environment, note: ssh keys should be in the .ssh folder)cd /opt/python/current/app/
(to change to django app folder)source ../env activate
(activates the virtual environment)python manage.py createsuperuser
(to create superuser in aws application)
- Install awsebcli
- Using HEROKU
- Create Environment variables in production environment
- SECRET_KEY=<secret-key for django>
- PROJECT_DOMAIN=<root domain>
- DEP_NAME=<project>/master (e.g. for master)
- REDIS_URL=<redis url for cache>
- SENDWITHUS_API_KEY=<sendwithus-key>
- SENTRY_DSN=<sentry-dsn-url>
- CLOUDINARY_URL=<cloudinary-url>
- Database settings:
- Heroku: DATABASE_URL=<database-url>
- AWS:
- RDS_DB_NAME=<RDS_DB_NAME>
- RDS_USERNAME=<RDB_USERNAME>
- RDS_PASSWORD=<RDS_PASSWORD>
- RDS_HOSTNAME=<RDS_HOSTNAME>
- RDS_PORT=<RDS_PORT>
- Requirements settings: create a folder called requirements and inside that create three files
That's it ENJOY !