Serverless Django with PostgreSQL with Cloud SQL and Cloud Run

Below is a reference guide for the Serverless Django project.

I created it as a configuration overview for using Django on Google Cloud Run as a Serverless Application using Google Cloud SQL's postgres database.

Keep in mind that there's a lot of context missing here which is why I recommend you go through the project first and then reference this for every future project.

Local Reference Paths

My local username: (type whoami in Terminal/PowerShell):

cfe

Local project path (using macOS):

/Users/cfe/dev/serverless-django

Local SQL Proxy Executable:

/Users/cfe/cloud_sql_proxy

Local IAM Service Account Credential File Path:

/Users/cfe/dev/serverless-django/db-proxy-iam.json

Google Cloud Reference

The below reference is connected to my project directly. Change yours accordingly.

Google cloud project id <gcp-project-id>:

serverless-cfe

Container tag

serverless-django

Container registry hostname (options):

gcr.io

Container Registery Image (<registry-hostname>/<gcp-project-id>/<container-tag>):

gcr.io/serverless-cfe/serverless-django

Cloud Run service name:

serverless-django-run

Cloud SQL Instance Name:

serverless-django-sql-01

Cloud Run & SQL Region <gcp-region>:

us-west1

Cloud SQL Instance Connection Name ( <gcp-project-id>:<gcp-region>:<cloud-sql-instance-name>):

serverless-cfe:us-west1:serverless-django-sql-01

You can also find the instance connection name on the Cloud SQL Instance itself.

Database name (in the Cloud SQL Instance):

serverlessdjangodb

IAM Service Account Email:

django-sql-service-account@serverless-cfe.iam.gserviceaccount.com

Django Project Reference

Project name django-admin startproject <project-name>

serverless_django

Django base project tree

bash

├── Dockerfile
├── Pipfile
├── db-proxy-iam.json
├── db.sqlite3
├── manage.py
├── scripts
│   ├── build.sh
│   ├── db_proxy.sh
│   ├── deploy.sh
│   ├── docker_run.sh
│   ├── entrypoint.sh
│   └── gcp.sh
└── serverless_django
    ├── __init__.py
    ├── asgi.py
    ├── settings
    │   ├── __init__.py
    │   ├── local.py
    │   └── production.py
    ├── urls.py
    └── wsgi.py

As you can see, we've moved settings.py to settings/local.py and created settings/production.py. This is to keep our environments separate. Read more on staging django.

Django Database Settings

Be sure to run pipenv install psycopg2-binary

# settings/local.py
DATABASES = {
        'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'serverless_django_local_db',
        'USER': 'serverless_django_local_user',
        'PASSWORD': '<your-password>',
        'HOST': '127.0.0.1',
        'PORT': 6543,
    }
}

PORT: 6543 is used since we'll be using a local proxy to our Google Cloud SQL database.

# settings/production.py

INSTANCE_CONNECTION_NAME = os.environ("INSTANCE_CONNECTION_NAME)
DATABASES = {
        'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'serverless_django_prod_db',
        'USER': 'serverless_django_prod_user',
        'PASSWORD': '<your-password>',
        'HOST': f'/cloudsql/{INSTANCE_CONNECTION_NAME}'
    }
}

INSTANCE_CONNECTION_NAME is set as an environment variable based on the Cloud SQL Instance Connection Name that is set we run gcloud run deploy

Automation & Deployment Scripts

All of the below scripts live in the scripts/ directory at the root of my Django project (ie next to manage.py).

Windows Users: a lot of these scripts can be run within PowerShell, just change .sh to .ps1 and you should be able to run it like .\scripts\build.ps1

scripts/build.sh

gcloud builds submit --tag gcr.io/serverless-cfe/serverless-django .

scripts/db_proxy.sh

This is used for running a local proxy into our database. Below we use the tcp port 6543 to avoid any conflicts with any local version of postgresql running.

exec /Users/cfe/cloud_sql_proxy-instances=serverless-cfe:us-west1:serverless-django-sql-01=tcp:6543 -credential_file=/Users/cfe/dev/serverless-django/db-proxy-iam.json

scripts/deploy.sh

gcloud builds submit --tag gcr.io/serverless-cfe/serverless-django .

gcloud run deploy serverless-django-run --image gcr.io/serverless-cfe/serverless-django \
--platform managed \
--region us-west1  \
--allow-unauthenticated \
--add-cloudsql-instances serverless-django-sql-01 \
--update-env-vars INSTANCE_CONNECTION_NAME="serverless-cfe:us-west1:serverless-django-sql-01" \
--service-account django-sql-service-account@serverless-cfe.iam.gserviceaccount.com

scripts/docker_run.sh

Build and run project locally. This simplifies testing. Setting the PORT environment variable is required since Google Run sets it for us as well.

docker build -t serverless-django -f Dockerfile .
docker run --env PORT=8888 -it -p 8888:8888 serverless-django

scripts/entrypoint.sh

#!/bin/bash

/usr/local/bin/gunicorn serverless_django.wsgi:application --bind "0.0.0.0:$PORT" --workers 3 --env DJANGO_SETTINGS_MODULE=serverless_django.settings.production

Postgres on Cloud SQL

Assuming you have Cloud SQL Proxy setup locally and a Cloud SQL Instance we'll create our databases and database users with the following:

Start your database proxy (scripts/db_proxy.sh created above):
./scripts/db_proxy.sh
Verify postgres is installed locally (or install via homebrew with brew install postgres):
$ psql -V psql (PostgreSQL) 12.2
Login to Cloud SQL PostgreSQL via
psql -U postgres -h 127.0.0.1 -p 6543 -d postgres
Create users & databases for our projects

Production database
CREATE ROLE serverless_django_prod_db WITH LOGIN PASSWORD '<your-password>'; ALTER ROLE serverless_django_prod_user CREATEDB; create database serverless_django_prod_db; grant all privileges on database serverless_django_prod_db to serverless_django_prod_user;
Development database
CREATE ROLE serverless_django_local_db WITH LOGIN PASSWORD '<your-password>'; ALTER ROLE serverless_django_local_user CREATEDB; create database serverless_django_local_db; grant all privileges on database serverless_django_local_db to serverless_django_local_user;

Dockerfile

FROM python:3.8.2-slim

ENV APP_HOME /app
WORKDIR $APP_HOME

COPY . ./

RUN pip install pip pipenv django gunicorn --upgrade
RUN pipenv install --skip-lock --system --dev

CMD ["./scripts/entrypoint.sh"]

Resources:

Preparing your Django Application for Google Cloud Run. This guide was huge help even with some things changing.
Connecting to Cloud SQL from Cloud Run (GCP Docs)
Cloud SQL Proxy