Deployment#

✓
Documentation Verified Last checked: 2026-02-23 Reviewer: Christof Buchbender

Overview#

Deployments are driven by a causal build-state orchestrator in ci/check_builds.py, triggered via GitHub Actions (orchestrate-deploy.yml) and executed by Jenkins.

Repos are divided into deploy groups. Each group has its own gate check and its own Jenkins job — a docs rebuild never blocks or is blocked by a main-stack rebuild.

Deploy groups#

Group

Repos

Jenkins jobs

default

ops-db, ops-db-api, data-transfer, ops-db-ui

deploy-staging / deploy-production

docs

data-center-documentation

deploy-data-center-documentation-develop / deploy-data-center-documentation-production

Main-stack deploy (default group)#

  1. A developer pushes to one of the main-stack repos.

  2. GitHub Actions builds and pushes the Docker image, then calls notify-build-complete.yml which sends a repository_dispatch (type build-complete) to system-integration.

  3. orchestrate-deploy.yml runs (serialised per branch via a concurrency group):

    1. Update state — records sha, image_digest, and built_with in BUILD_STATE_JSON_<BRANCH>.

    2. Cascade — dispatches workflow_dispatch to immediate downstream repos so they rebuild against the new upstream image.

    3. Cross-group dispatch — dispatches update-submodules.yml in data-center-documentation for each cross_group_trigger entry in dependency-graph.yml.

    4. Gate check — resolves the triggering repo’s deploy group, then verifies every repo in that group is causally consistent (built_with matches current upstream digest). If all green, triggers the Jenkins job via its REST API.

  4. Jenkins pipeline (deploy_staging / deploy_production):

    • SSHs into input-b.{staging.}data.ccat.uni-koeln.de

    • Runs Alembic migrations (ops-db pipeline only)

    • docker compose pull + up -d (main stack compose file)

    • Smoke test

Docs deploy (docs group)#

The docs stack is independent — it has its own compose files (docker-compose.docs.input-b.yml / docker-compose.docs.staging.input-b.yml) and its own Jenkins jobs.

The trigger chain:

  1. Any of the four tracked upstream repos publishes a new image.

  2. The orchestrator fires a cross-group workflow_dispatch to data-center-documentation, targeting update-submodules.yml.

  3. update-submodules.yml bumps all tracked submodule pointers to their latest HEAD on the dispatched branch, then commits and pushes (if anything changed).

  4. The push triggers docker-docs.yml, which builds and pushes the docs image and calls notify-build-complete with the submodule SHAs as built_with_json.

  5. The orchestrator receives the build-complete event, updates state, and runs the gate check for the docs group. The gate uses source_commit comparison: built_with[dep] is compared against state[dep].sha (git commit SHA of each submodule pointer) rather than an image digest.

  6. If all four submodule SHAs match, the Jenkins docs deploy job is triggered.

Causal consistency model#

Build state is stored in two GitHub repository variables: BUILD_STATE_JSON_DEVELOP and BUILD_STATE_JSON_MAIN. Each per-repo entry has the form:

{
  "sha":          "<git commit SHA>",
  "image_digest": "sha256:...",
  "ts":           "2026-01-01T00:00:00+00:00",
  "built_with":   { "ops-db": "sha256:..." }
}

The built_with_type field in ci/dependency-graph.yml controls what the gate check compares:

Type

Gate compares built_with[dep] against

runtime

state[dep].image_digest

base_image

state[dep].image_digest

source_commit

state[dep].sha (used by docs: submodule pointer vs upstream git SHA)

Local Development#

git clone <repository_url>
cd system-integration
git submodule update --init --recursive
cp .env.example .env
# Edit .env — set POSTGRES_PASSWORD, REDIS_PASSWORD, etc.
ccat update   # or: make start_main

Staging Environment#

Staging runs on input-b.staging.data.ccat.uni-koeln.de.

Jenkins deploy-staging / deploy-data-center-documentation-develop handle staging deploys automatically when the gate check passes on the develop branch.

To deploy manually:

ssh <user>@input-b.staging.data.ccat.uni-koeln.de
cd /opt/data-center/system-integration
docker compose -f docker-compose.staging.input-b.yml pull
docker compose -f docker-compose.staging.input-b.yml up -d

Production Deployment#

Production runs on input-b.data.ccat.uni-koeln.de.

Jenkins deploy-production / deploy-data-center-documentation-production handle production deploys automatically when the gate check passes on the main branch.

To deploy manually:

ssh <user>@input-b.data.ccat.uni-koeln.de
cd /opt/data-center/system-integration
# Main stack:
docker compose -f docker-compose.production.input-b.yml pull
docker compose -f docker-compose.production.input-b.yml up -d
# Docs (independent):
docker compose -f docker-compose.docs.input-b.yml pull
docker compose -f docker-compose.docs.input-b.yml up -d