CollabOps

Monorepo Pipeline

Monorepo CI/CD pipeline with parallel Frontend/Backend builds, tests, and security scans

A pipeline for monorepo projects that manage Frontend and Backend in a single repository. 14 Jobs run in parallel, utilizing complex DAG dependencies.

Full Code

name: monorepo-pipeline

triggers:
  push:
    branches: [main]

jobs:
  # ═══════════════════════════════════════
  # Phase 1: Source — Source code checkout
  # ═══════════════════════════════════════

  # Main repository checkout
  source-main:
    phase: source
    steps:
      - name: checkout-main
        uses: "collabops/checkout@v2"
        with:
          repo-url: "https://<collabops-host>/<workspace>/<repository>.git"

  # Shared dependency repository checkout (submodules, etc.)
  source-deps:
    phase: source
    steps:
      - name: checkout-deps
        uses: "collabops/checkout@v2"
        with:
          repo-url: ${{ vars.DEPS_REPO_SSH_URL }}
          ssh-key: ${{ secrets.GIT_SSH_PRIVATE_KEY }}
          path: /workspace/deps

  # ═══════════════════════════════════════
  # Phase 2: Deps — Dependency installation (Frontend/Backend in parallel)
  # ═══════════════════════════════════════

  # Frontend dependency installation
  deps-frontend:
    phase: deps
    needs: [source-main, source-deps]    # Both sources required
    steps:
      - name: install-frontend-deps
        image: node:18
        run: |
          cd /workspace/source/frontend
          npm ci

  # Backend dependency installation
  deps-backend:
    phase: deps
    needs: [source-main, source-deps]    # Runs in parallel with Frontend
    steps:
      - name: install-backend-deps
        image: python:3.11
        run: |
          cd /workspace/source/backend
          pip install -r requirements.txt

  # ═══════════════════════════════════════
  # Phase 3: Build — Build (Frontend/Backend in parallel)
  # ═══════════════════════════════════════

  build-frontend:
    phase: build
    needs: [deps-frontend]
    steps:
      - name: build-frontend
        image: node:18
        run: |
          cd /workspace/source/frontend
          npm run build

  build-backend:
    phase: build
    needs: [deps-backend]
    steps:
      - name: build-backend
        image: python:3.11
        run: |
          cd /workspace/source/backend
          python setup.py build

  # ═══════════════════════════════════════
  # Phase 4: Test — Tests + Security scans (all in parallel)
  # ═══════════════════════════════════════

  # Frontend unit tests
  test-frontend-unit:
    phase: test
    needs: [build-frontend]
    steps:
      - name: test-frontend
        image: node:18
        run: |
          cd /workspace/source/frontend
          npm test
        env:
          CI: "true"

  # Backend unit tests
  test-backend-unit:
    phase: test
    needs: [build-backend]
    steps:
      - name: test-backend
        image: python:3.11
        run: |
          cd /workspace/source/backend
          pytest tests/unit/

  # E2E tests — After both Frontend + Backend builds complete
  test-e2e:
    phase: test
    needs: [build-frontend, build-backend]
    steps:
      - name: e2e-tests
        image: node:18
        run: |
          cd /workspace/source
          npm run test:e2e

  # Frontend security scan
  security-frontend:
    phase: test
    needs: [build-frontend]
    steps:
      - name: security-scan-frontend
        image: node:18
        run: |
          cd /workspace/source/frontend
          npm audit --audit-level=high

  # Backend security scan
  security-backend:
    phase: test
    needs: [build-backend]
    steps:
      - name: security-scan-backend
        image: python:3.11
        run: |
          cd /workspace/source/backend
          pip audit

  # Container image security scan
  security-container:
    phase: test
    needs: [build-frontend, build-backend]
    services:
      - docker
    steps:
      - name: container-scan
        image: docker:27.5-cli
        run: |
          docker build -t myapp:scan .
          # Scan container vulnerabilities with Trivy
          trivy image myapp:scan

  # ═══════════════════════════════════════
  # Phase 5: Deploy — Staging → Production sequential deployment
  # ═══════════════════════════════════════

  # Staging deployment — After all tests + security scans pass
  deploy-staging:
    phase: deploy
    needs:
      - test-frontend-unit
      - test-backend-unit
      - test-e2e
      - security-frontend
      - security-backend
      - security-container
    steps:
      - name: deploy-staging
        run: kubectl apply -f k8s/staging/

  # Production deployment — After staging verification
  deploy-production:
    phase: deploy
    needs: [deploy-staging]
    if: "collabops.ref == 'refs/heads/main' && collabops.event_name == 'push'"
    steps:
      - name: deploy-production
        run: kubectl apply -f k8s/production/

Execution Flow

source-main ──┬── deps-frontend ── build-frontend ──┬── test-frontend-unit ──┐
              │                                     ├── security-frontend ───┤
              │                                     │                        │
source-deps ──┤                                     ├── test-e2e ────────────┤
              │                                     │                        ├── deploy-staging ── deploy-production
              │                                     ├── security-container ──┤
              │                                     │                        │
              └── deps-backend ─── build-backend ───┼── test-backend-unit ──┤
                                                    └── security-backend ───┘

Key Points

Maximized parallel execution: Frontend and Backend independently progress through deps → build → test

E2E + Security scans: Cross-dependencies that require both Frontend/Backend builds to complete before running

6 gates: All 6 Jobs must succeed before staging deployment

Sequential deployment: Safe deployment in staging → production order

Table of Contents