Building a GitHub Actions CI/CD Pipeline

Building a GitHub Actions CI/CD Pipeline

This is a writeup of how I built a GitHub Actions-based CI/CD pipeline to support the rapid deployment cycle of the XGen platform.

Pipeline Structure

Push to main → Lint/Test → Build Docker Image → Push to Registry → Deploy to Server

CI Workflow: Tests and Linting

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  lint-and-test:
    runs-on: ubuntu-latest

    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_DB: xgen_test
          POSTGRES_USER: test
          POSTGRES_PASSWORD: test
        ports:
          - 5432:5432

      redis:
        image: redis:7
        ports:
          - 6379:6379

    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'

      - name: Install dependencies
        run: |
          cd backend
          pip install -r requirements.txt
          pip install pytest pytest-asyncio ruff

      - name: Lint with Ruff
        run: |
          cd backend
          ruff check .

      - name: Run tests
        env:
          DATABASE_URL: postgresql+asyncpg://test:test@localhost:5432/xgen_test
          REDIS_URL: redis://localhost:6379
        run: |
          cd backend
          pytest tests/ -v --asyncio-mode=auto

CD Workflow: Docker Build and Deploy

# .github/workflows/cd.yml
name: CD

on:
  push:
    branches: [main]

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    needs: [lint-and-test]

    steps:
      - uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to Docker Registry
        uses: docker/login-action@v3
        with:
          registry: ${{ secrets.DOCKER_REGISTRY }}
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: Build and push backend
        uses: docker/build-push-action@v5
        with:
          context: ./backend
          push: true
          tags: |
            ${{ secrets.DOCKER_REGISTRY }}/xgen-backend:latest
            ${{ secrets.DOCKER_REGISTRY }}/xgen-backend:${{ github.sha }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

      - name: Deploy to server
        uses: appleboy/ssh-action@v1
        with:
          host: ${{ secrets.SERVER_HOST }}
          username: ${{ secrets.SERVER_USER }}
          key: ${{ secrets.SSH_PRIVATE_KEY }}
          script: |
            cd /opt/xgen
            docker compose pull
            docker compose up -d --build
            docker image prune -f

Per-Environment Deployment Strategy

Development, staging, and production environments are managed separately.

# Branch-to-environment deployment mapping
# develop → staging server
# main → production server

  deploy:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        include:
          - branch: develop
            server: staging
            env_file: .env.staging
          - branch: main
            server: production
            env_file: .env.production

Dockerfile Optimization

Multi-stage builds and layer caching keep build times short.

# backend/Dockerfile
FROM python:3.11-slim AS builder

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt

FROM python:3.11-slim

WORKDIR /app
COPY --from=builder /install /usr/local
COPY . .

EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]

Health Checks and Zero-Downtime Deployment

services:
  backend:
    image: xgen-backend:latest
    deploy:
      update_config:
        parallelism: 1
        delay: 10s
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

With this CI/CD pipeline in place, deployments complete automatically within roughly 5 minutes of a code push. Over 303 commits, the value of this pipeline was apparent every single time. Deployment frequency increased by more than 3× compared to manual deploys, and deployment-related mistakes dropped to nearly zero.