DevSecOps

Security tooling integrated into CI/CD finds vulnerabilities before they reach production — and at a fraction of the cost of post-deployment remediation. The key insight: security gates must be fast (developers won't wait 10 minutes for a security check) and low-noise (too many false positives trains teams to ignore findings). This skill covers building a practical DevSecOps pipeline that developers will actually use.

Core Mental Model

Shift-left security means moving security checks earlier in the development lifecycle — pre-commit for credential scanning, PR time for SAST and dependency checks, build time for container scanning, and staging for DAST. The investment per vulnerability found drops by 100x as you move earlier. Each layer has a different purpose: pre-commit stops the most obvious issues; SAST finds code-level bugs; SCA finds vulnerable dependencies; container scanning finds OS and package vulnerabilities; DAST finds runtime behavior issues that static analysis misses.

Pre-Commit Hooks

# Install pre-commit framework
pip install pre-commit

# .pre-commit-config.yaml — place in project root
cat > .pre-commit-config.yaml << 'EOF'
repos:
  # Credential/secret detection
  - repo: https://github.com/Yelp/detect-secrets
    rev: v1.5.0
    hooks:
      - id: detect-secrets
        args: ['--baseline', '.secrets.baseline']
        # Create baseline: detect-secrets scan > .secrets.baseline
  
  # AWS-specific credential scanning
  - repo: https://github.com/awslabs/git-secrets
    rev: 80230afa8c8bdeac766a0fece36f95ffaa0be778
    hooks:
      - id: git-secrets

  # General security linting
  - repo: https://github.com/PyCQA/bandit
    rev: 1.7.8
    hooks:
      - id: bandit
        args: ['-ll', '--skip', 'B101']  # -ll = medium+ only; skip assert warning
        exclude: tests/

  # Terraform IaC security
  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.90.0
    hooks:
      - id: terraform_tfsec
      - id: terraform_checkov

  # Standard pre-commit checks
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.6.0
    hooks:
      - id: no-commit-to-branch
        args: ['--branch', 'main', '--branch', 'production']
      - id: detect-private-key
      - id: check-added-large-files
        args: ['--maxkb=500']  # Catches accidentally committed model files
EOF

pre-commit install
pre-commit install --hook-type commit-msg  # For conventional commits

SAST in CI: Semgrep

# .github/workflows/sast.yml
name: SAST Scan

on:
  pull_request:
  push:
    branches: [main]

jobs:
  semgrep:
    name: Semgrep SAST
    runs-on: ubuntu-latest
    container:
      image: semgrep/semgrep
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Run Semgrep
        env:
          SEMGREP_APP_TOKEN: ${{ secrets.SEMGREP_APP_TOKEN }}
        run: |
          semgrep ci \
            --config=auto \            # Use all rulesets matching your language
            --config=p/security-audit \
            --config=p/owasp-top-ten \
            --error \                  # Exit non-zero on findings (blocks PR)
            --severity=ERROR \         # Only block on ERROR severity
            --json-output=semgrep-results.json
      
      - name: Upload SARIF results
        uses: github/codeql-action/upload-sarif@v3
        if: always()  # Upload even on failure for PR annotations
        with:
          sarif_file: semgrep-results.json

# Custom Semgrep rule — SQL injection pattern
rules:
  - id: sql-injection-string-format
    patterns:
      - pattern: |
          $DB.execute("..." % ...)
      - pattern: |
          $DB.execute(f"...{...}...")
      - pattern: |
          $DB.execute("..." + ...)
    message: |
      Potential SQL injection via string formatting. Use parameterized queries:
      db.execute("SELECT * FROM users WHERE id = %s", (user_id,))
    languages: [python]
    severity: ERROR
    metadata:
      cwe: "CWE-89"
      owasp: "A03:2021"
  
  - id: hardcoded-secret-in-test
    pattern: |
      password = "..."
    message: "Hardcoded password detected. Use environment variables or fixtures."
    languages: [python, javascript, typescript]
    severity: WARNING
    paths:
      exclude:
        - tests/fixtures/

SCA: Dependency Scanning

# GitHub Dependabot (automatic PR creation for vulnerable deps)
# .github/dependabot.yml
version: 2
updates:
  - package-ecosystem: "pip"
    directory: "/"
    schedule:
      interval: "weekly"
    open-pull-requests-limit: 10
    groups:
      security-patches:
        patterns: ["*"]
        update-types: ["patch"]

  - package-ecosystem: "npm"
    directory: "/frontend"
    schedule:
      interval: "weekly"
    ignore:
      - dependency-name: "next"
        versions: [">=15"]  # Pin major version manually

# Snyk for deeper SCA analysis (license, reachability)
  snyk-scan:
    name: Snyk Dependency Scan
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Snyk Python dependencies
        uses: snyk/actions/python@master
        env:
          SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
        with:
          args: |
            --severity-threshold=high
            --fail-on=upgradable
            --sarif-file-output=snyk-results.sarif
      
      - name: Upload Snyk SARIF
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: snyk-results.sarif

Container Scanning with Trivy

trivy-scan:
    name: Container Security Scan
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Build container image
        run: docker build -t ${{ github.repository }}:${{ github.sha }} .
      
      - name: Run Trivy vulnerability scanner
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: '${{ github.repository }}:${{ github.sha }}'
          format: 'sarif'
          output: 'trivy-results.sarif'
          severity: 'CRITICAL,HIGH'
          exit-code: '1'               # Block on CRITICAL/HIGH
          ignore-unfixed: true         # Don't fail on no-fix-available vulns
          vuln-type: 'os,library'
          security-checks: 'vuln,secret,config'
      
      - name: Upload Trivy SARIF
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: 'trivy-results.sarif'

# Trivy in a Dockerfile for reproducible local scans
# trivy image --severity CRITICAL,HIGH --exit-code 1 myapp:latest

SBOM Generation

sbom:
    name: Generate SBOM
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Generate SBOM with Syft
        uses: anchore/sbom-action@v0
        with:
          image: '${{ github.repository }}:${{ github.sha }}'
          format: cyclonedx-json       # Industry standard format
          output-file: sbom.cyclonedx.json
          artifact-name: sbom-${{ github.sha }}.json
      
      - name: Scan SBOM with Grype
        uses: anchore/scan-action@v3
        with:
          sbom: sbom.cyclonedx.json
          fail-build: true
          severity-cutoff: critical

# CLI: syft packages myapp:latest -o cyclonedx-json > sbom.json
# CLI: grype sbom:sbom.json --fail-on critical

Secrets Management

# NEVER commit secrets to code — not even in "private" repos

# ❌ WRONG — these are all discoverable in git history forever
API_KEY = "sk-1234567890abcdef"
os.environ["DB_PASSWORD"] = "mysecretpassword"

# ✅ CORRECT patterns:

# 1. AWS Secrets Manager (for applications running on AWS)
import boto3
import json

def get_secret(secret_name: str, region: str = "us-east-2") -> dict:
    client = boto3.client("secretsmanager", region_name=region)
    response = client.get_secret_value(SecretId=secret_name)
    return json.loads(response["SecretString"])

db_creds = get_secret("prod/myapp/postgres")
# IAM role grants access — no credentials needed in code

# 2. GitHub Actions OIDC (no static credentials for CI/CD)
# In your GitHub Actions workflow:
permissions:
  id-token: write  # Required for OIDC

steps:
  - name: Configure AWS credentials via OIDC
    uses: aws-actions/configure-aws-credentials@v4
    with:
      role-to-assume: arn:aws:iam::123456789:role/github-actions-role
      aws-region: us-east-2
      # NO static access keys needed — OIDC JWT proves GitHub identity

# 3. HashiCorp Vault for multi-cloud secrets
import hvac

client = hvac.Client(url="https://vault.company.com")
client.auth.aws.iam_login(role="myapp-prod")  # AWS IAM auth

secret = client.secrets.kv.v2.read_secret_version(
    path="myapp/database",
    mount_point="secret",
)
db_password = secret["data"]["data"]["password"]

IaC Security Scanning

# tfsec: Terraform security scanner
tfsec ./terraform \
  --minimum-severity HIGH \
  --format sarif \
  --out tfsec-results.sarif \
  --exclude aws-s3-no-public-access  # Exclude if intentionally public

# checkov: Multi-framework IaC scanner (Terraform, CloudFormation, Kubernetes, Dockerfile)
checkov -d ./terraform \
  --framework terraform \
  --check CKV_AWS_20,CKV_AWS_19 \  # Specific checks only
  --compact \
  --output sarif > checkov-results.sarif

# GitHub Actions IaC scanning
  iac-scan:
    steps:
      - name: tfsec
        uses: aquasecurity/[email protected]
        with:
          working_directory: terraform/
          minimum_severity: HIGH
          format: sarif
          sarif_file: tfsec-results.sarif
      
      - name: Checkov
        uses: bridgecrewio/checkov-action@master
        with:
          directory: .
          framework: terraform,dockerfile,kubernetes
          output_format: sarif
          output_file_path: checkov-results.sarif
          soft_fail: false  # Fail the build on violations

DAST with OWASP ZAP

dast:
    name: DAST Scan (Staging Only)
    environment: staging
    if: github.ref == 'refs/heads/main'  # Only on main, after staging deploy
    
    steps:
      - name: Wait for staging deploy
        run: sleep 60
      
      - name: ZAP API Scan
        uses: zaproxy/[email protected]
        with:
          target: 'https://staging.myapp.com/api/openapi.json'
          rules_file_name: '.zap/rules.tsv'
          cmd_options: '-a -j'  # -a: alert on new; -j: AJAX spider
          fail_action: true     # Fail on Medium+ findings
          artifact_name: zap-report

Complete DevSecOps GitHub Actions Pipeline

# .github/workflows/security.yml
name: Security Pipeline

on:
  pull_request:
  push:
    branches: [main]

permissions:
  contents: read
  security-events: write  # For SARIF upload
  pull-requests: write    # For PR comments

jobs:
  # Fast checks first (fail fast principle)
  secrets-scan:
    name: Secrets Detection
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Full history for secret scanning
      - uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

  sast:
    name: Static Analysis
    runs-on: ubuntu-latest
    needs: secrets-scan
    container: semgrep/semgrep
    steps:
      - uses: actions/checkout@v4
      - run: semgrep ci --config=auto --error --severity=ERROR
        env:
          SEMGREP_APP_TOKEN: ${{ secrets.SEMGREP_APP_TOKEN }}

  sca:
    name: Dependency Scan
    runs-on: ubuntu-latest
    needs: secrets-scan
    steps:
      - uses: actions/checkout@v4
      - uses: snyk/actions/python@master
        env:
          SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
        with:
          args: --severity-threshold=high

  container:
    name: Container Scan
    runs-on: ubuntu-latest
    needs: [sast, sca]
    steps:
      - uses: actions/checkout@v4
      - run: docker build -t app:${{ github.sha }} .
      - uses: aquasecurity/trivy-action@master
        with:
          image-ref: app:${{ github.sha }}
          severity: CRITICAL,HIGH
          exit-code: 1

  iac:
    name: IaC Security
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: aquasecurity/[email protected]
        with:
          minimum_severity: HIGH

Anti-Patterns

❌ Blocking CI on everything including low-severity findings
A pipeline that blocks on 500 low-severity findings is a pipeline developers bypass. Block on Critical/High; warn on Medium; informational on Low.

❌ No baseline for false positives
Semgrep and other SAST tools will have false positives. Without a way to suppress them (inline comments, baseline files), developers will route around the tooling entirely.

❌ Running DAST against production
DAST actively probes for vulnerabilities — it will trigger alerts, create test data, and potentially cause errors. Always run against staging/test environments.

❌ Storing secrets in GitHub Actions environment variables in plain text
Use GitHub Secrets for sensitive values. Never echo secrets in logs. Prefer OIDC over static credentials.

❌ Scanning only on pull requests
Main branch should also be scanned. Dependencies introduced through merge commits or direct pushes need scanning too.

Quick Reference

Security gate thresholds (recommended):
  Block PR:  CRITICAL and HIGH severity findings with available fixes
  Warn:      MEDIUM severity, no-fix-available HIGH
  Inform:    LOW and INFORMATIONAL

Pipeline stage latency targets:
  Pre-commit (local)    → <3 seconds
  SAST (Semgrep)        → <3 minutes
  SCA (Snyk)            → <2 minutes
  Container scan (Trivy) → <5 minutes
  SBOM generation       → <2 minutes
  DAST (ZAP)            → 10-30 minutes (staging only)

Tool selection by language:
  Python    → Bandit (simple), Semgrep (advanced), Safety (deps)
  Node.js   → ESLint-security, Semgrep, npm audit, Snyk
  Go        → gosec, Semgrep, govulncheck
  Java      → SpotBugs + Find-Sec-Bugs, OWASP Dependency-Check
  Multi     → Semgrep + Snyk + Trivy (works for everything)

SBOM formats:
  CycloneDX  → Preferred (richer metadata, OWASP standard)
  SPDX       → Linux Foundation standard, required by some regulations
  Both       → Generate both for compliance coverage