Django on AWS Lambda

Tl;dr

This blog is now officially Serverless -- no EC2, no ELB, no RDS*, no ongoing server fees.

Below I go through how my traditional Django cloud architecture used to be like and how it has changed with the Serverless revolution, and I wrap up with the easy steps involved in deploying the new serverless Django using ZAPPA - Serverless Python Web Services)

Traditional Django on AWS

(the cloud architect's way)

How do you host a Django application on AWS? Well, there are a number of options. My first approach was to set up an EC2 instance and an RDS instance for the database. Of course, you then need to manage your instances, make sure you keep on top of the security patches etc. Besides, you can't just have 1 instance, what if it fails? You need a Load Balancer for high availibility and an Auto Scaling Group to handle traffic spikes. That's crucial, right?

The "proper" website setup on AWS includes an Auto Scaling Group EC2 instances across at least 2 Availability Zones, behind and an Elastic Load Balancer linked to an RDS database with Multi-AZ for redundancy and automatic backups. All behind a Cloudfront (CDN).

simple-web-host-arch A "simplification" of the AWS "WEB-APPLICATION
HOSTING" reference architecture

Serverless Django

With the age of serverless upon us you no longer need to pay for any servers to run a small web application. AWS Lambda charges only for the views you get, not for the time the servers sit idle.

In terms of costs, the majority of the costs of the above model, would be, in order, the EC2 instances, the RDS instance(s) and the ELB. In AWS's world of serverless, API Gateway is the entrypoint that redirects the incomming requests to the Lambda functions. As for the Auto Scaling group, Lambda will self manage and scale to match the number of simultaneous requests at any given time.

lambda-web-host-arch API Gateway replacing the ELB, and Lambda in place of auto scaling EC2 instances

You may notice I have left the multi-AZ RDS instances in place. Django is quite tightly coupled a relational database, and while I don't consider RDS serverless, these servers are fully managed by AWS.

[*] "Cheapskate-Protip" In my intro I mentioned my blog has "no RDS", yet I kept it in the recommended architecture. This is because Django, does require a database to keep its magical orm, but as a happy accident, I realised, my blog does not have any state that needs to be kept between requests. I'm quite happy to keep my drafts on my laptop, in an Sqlite3 database, and package and deploy the database with the code as my "publish" step. Side effects include: no ongoing RDS fees, and my database is zipped up and stored on S3 with each deployment. #automaticbackups

The Components

As with most solutions on AWS, you need to touch on a number of AWS services to put this together. The diagram shows components such as:

Route 53 (for DNS)
S3 (for Static content)
CloudFront (for CDN)
API Gateway (for managing requests / Load Balancing-ish)
AWS Lambda (for executing your code)
RDS (for DB)

In additon to those there are the fundamental AWS services that often go unnoticed on diagrams -- IAM (for permission policies), and Certificate Manager (for secure https traffic) -- as well as some docker magic for handling your dependencies.

The easy way

This might feel like a lot of components to get familiar with, but thankfully, you don't really have to from the beginning.

Tools like ZAPPA and Serverless Framework can help you put it all together with minimal effort. I'll cover the required steps in Zappa below, but the process is much the same using other similar tools. Both produce the same end state, doing most of the heavy lifting for you.

The steps are:

Install and setup AWS CLI (pip install awscli; aws configure).
Install and initialize Zappa (pip install zappa; zappa init)
Deploy your application (zappa deploy)
- This wraps up your python dependencies, deploys it on Lambda and configures the API Gateway entrypoint
Setup and certify a domain
1. Create a hosted zone (aws route53 create-hosted-zone ...)
2. Request a certificate (aws acm request-certificate ...)
3. Add "domain" and "certificate_arn" to zappa_settings.json
4. Certify (zappa certify)
  - These steps complete your setup, by adding a route 53 and a CloudFront distribution to serve your code on own domain (over https://).

That's it**, your Django application is hosted on Lambda.

** You will need to correct your ALLOWED_HOSTS Django setting to include your new domain.

Appendix - Django on AWS Lambda Scratchpad

See below for exact commands to run complete each step.

1. Setup AWS CLI profile

$ pip install awscli
...

$ aws configure --profile=awsuser
AWS Access Key ID [None]: XXXXXXXXXXXXXXXXXXXX  
AWS Secret Access Key [None]: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  
Default region name [None]: ap-southeast-2  
Default output format [None]: json

2. Install and configure Zappa

$ pip install zappa
...

$ zappa init # Repeatedly press enter to accept all defaults
...  
Okay, here's your zappa_settings.json:
{  
    "dev": {
        "aws_region": "ap-southeast-2",
        "django_settings": "mysite.settings",
        "profile_name": "awsuser",
        "project_name": "mysite",
        "runtime": "python3.6",
        "s3_bucket": "zappa-xxxxxxxxx"
    }
}
...

3. Deploy your application

$ zappa deploy dev
...
Deployment complete!: https://xxxxxxxxxx.execute-api.ap-southeast-2.amazonaws.com/dev

At this point your Django application is deployed and accessible on the URL printed from the previous command. But let's move on to a domain of our choise

4.0 Choose and buy your domain

First, choose an available domain and buy it if you don't already own it, then:

export DOMAIN=example.com

4. Create a Route 53 Hosted Zone

$ aws route53 create-hosted-zone \  
  --name $DOMAIN \
  --caller-reference $(date +%Y-%m-%d)

5. Request a certificate from Certificate Manager

$ aws acm request-certificate \
  --region us-east-1 \
  --domain-name $DOMAIN \
  --validation-method EMAIL \
  --subject-alternative-names www.$DOMAIN \
  --domain-validation-options DomainName=$DOMAIN,ValidationDomain=$DOMAIN
{
    "CertificateArn": "arn:aws:acm:us-east-1:000000000000:certificate/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}

6. Add domain and certifacate to zappa_setting.json

Edit zappa_settings.json providing the domain (from step 4) and the certificate_arn (from step 5) so that Zappa can configure API Gateway and Cloudfront for you.

{  
    "dev": {  
        "aws_region": "ap-southeast-2",  
        "django_settings": "mysite.settings",  
        "profile_name": "davur",  
        "project_name": "mysite",  
        "runtime": "python3.6",  
        "s3_bucket": "zappa-xxxxxxxxx",  
        "domain": "www.example.com",
        "certificate_arn": "arn:aws:acm:us-east-1:000000000000:certificate/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
    }  
}

7. Let Zappa tie it all together

$ zappa certify

That's it.

For future updates to you application you need only run zappa update.