Docker + Django + Redis + Celery Complete Setup Guide

Docker + Django + Redis + Celery Complete Setup Guide

This is a real production-level architecture.

Architecture

  • Django → Web App

  • PostgreSQL → Database

  • Redis → Broker

  • Celery → Background tasks

  • Nginx → Reverse proxy

Key Technologies

  • Django

  • Redis

  • Celery

  • PostgreSQL

docker-compose.yml

version: "3.9"

services:
  web:
    build: .
    command: gunicorn project.wsgi:application --bind 0.0.0.0:8000
    volumes:
      - .:/code
    ports:
      - "8000:8000"
    depends_on:
      - db
      - redis

  db:
    image: postgres:15
    environment:
      POSTGRES_DB: mydb
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: mypassword

  redis:
    image: redis:7

  celery:
    build: .
    command: celery -A project worker -l info
    depends_on:
      - redis
      - db

Django Celery Config

# project/celery.py

from celery import Celery
import os

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings")

app = Celery("project")
app.config_from_object("django.conf:settings", namespace="CELERY")
app.autodiscover_tasks()

settings.py
CELERY_BROKER_URL = "redis://redis:6379/0"
Now you can run:
docker-compose up --build


Docker Production Deployment on AWS
When deploying to cloud, common options:
  • Amazon ECS
  • Amazon EKS
  • AWS Elastic Beanstalk
  • Amazon EC2
Simple Production Setup (EC2 + Docker)
Step 1: Launch EC2
  • Ubuntu instance
  • Open ports 80 & 443
Step 2: Install Docker
sudo apt update
sudo apt install docker.io
Step 3: Pull Image
docker pull yourdockerhub/django-app
Step 4: Run with Nginx Reverse Proxy
Use:
  • Gunicorn inside container
  • Nginx on host or container
  • HTTPS via Certbot
Production Best Practices
  • Use environment variables
  • Use secrets manager
  • Enable logging
  • Use health checks
  • Auto-scaling group
  • Use RDS instead of container DB
Dockerfile Optimization Deep Dive
Bad Dockerfile
FROM python:3.12
COPY . .
RUN pip install -r requirements.txt
CMD ["python", "app.py"]
Problems:
  • Large image
  • No caching
  • Copies everything
Optimized Version
FROM python:3.12-slim

ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

USER nobody

CMD ["gunicorn", "project.wsgi:application", "--bind", "0.0.0.0:8000"]
Multi-Stage Example
FROM python:3.12 as builder

WORKDIR /install
COPY requirements.txt .
RUN pip install --prefix=/install -r requirements.txt

FROM python:3.12-slim
WORKDIR /app
COPY --from=builder /install /usr/local
COPY . .
CMD ["python", "app.py"]
Key Optimization Tips
  • Order layers properly
  • Use slim images
  • Use specific versions
  • Clean apt cache
  • Reduce build context
  • Avoid root user
  • Use healthcheck
Example:
HEALTHCHECK CMD curl --fail http://localhost:8000 || exit 1

Comments

Post a Comment

Popular posts from this blog

Database Integration in FastAPI (SQLAlchemy CRUD)

  Introduction So far, we’ve built APIs without storing data.But real-world applications need a database to: Store user data. Manage products. Persist application state. In this blog, we’ll connect FastAPI with a database using SQLAlchemy and perform CRUD operations . What is SQLAlchemy? SQLAlchemy is a powerful Python ORM (Object Relational Mapper) that allows you to: Interact with databases using Python code. Avoid writing raw SQL queries. Work with models instead of tables. Step 1: Install Dependencies pip install sqlalchemy (For SQLite, no extra driver needed) Project Structure Update app/ ├── main.py ├── database/ │ ├── connection.py ├── models/ │ ├── item.py ├── schemas/ │ ├── item.py ├── routes/ │ ├── item.py Step 2: Database Connection app/database/connection.py from sqlalchemy import create_engine from sqlalchemy . orm import sessionmaker , declarative_base DATABASE_URL = "sqlite:///./test.db" engine = create_engine ( DAT...
Read more

Middleware & CORS in FastAPI

  Introduction As your API grows, you may need to: Log requests Modify responses Handle cross-origin requests This is where Middleware and CORS (Cross-Origin Resource Sharing) come into play.They help control how requests and responses flow through your application. What is Middleware? Middleware is a function that runs: Before a request reaches your route After a response is returned Think of it as a pipeline layer between client and server. Request Flow with Middleware Client → Middleware → Route → Middleware → Response Creating Custom Middleware from fastapi import FastAPI , Request import time app = FastAPI () @ app . middleware ( "http" ) async def log_requests ( request : Request , call_next ): start_time = time . time() response = await call_next ( request ) process_time = time . time() - start_time print ( f"Request: { request . url} | Time: { process_time } " ) return response What This Middle...
Read more

Python Data Handling

Python Data Handling for Beginners In Python, data handling means storing, changing, and using data in different ways. In this post, you will learn: Lists Tuples Dictionaries Sets String Methods Type Conversion 1. Python Lists :Lists store many values in one variable.                fruits = ["apple", "banana", "mango"]                                                                       Method                          Use                                            append()          ...
Read more