# Client setup — SSH with step-ca certificates This page is a **how-to** for setting up your laptop or workstation to authenticate to CCAT hosts with a step-ca-issued SSH certificate. The short version of this is also covered in {doc}`ssh-access` (the operator-facing entry point); this document is the longer manual walkthrough that the `ccat ssh setup` automation collapses into one command. For the design context — why we chose this model, how the GitHub-team gate works, what cert lifetimes mean — see {doc}`background/ca-architecture`. ```{admonition} Who this page is for :class: note You're a partner or new user whose source CIDR is already on the CCAT proxy allowlist (Uni Köln `/16` by default). If it isn't, the CCAT operator runs *Adding a partner subnet* in {doc}`ca-day-to-day` first — you don't. Off-network laptops have separate options below in *Bootstrapping from off-network*. ``` ```{tip} The entire setup below is automated by a single command: pip install -e . # if you haven't installed the ccat CLI yet ccat ssh setup It is idempotent — safe to run again if you want to update the renewal helper or re-issue a certificate. Run `ccat ssh status` afterwards to verify everything is in order. The manual steps below document what the command does under the hood. ``` This walkthrough takes about 10 minutes. After setup, your daily workflow is just `ssh ` — the certificate renews automatically when it expires. ## Prerequisites - Member of the `ccatobs/datacenter` GitHub team (Dex rejects everyone else at the authentication step). - A web browser on the machine you SSH from (the GitHub OAuth flow opens a browser tab). ## Step 1 — Install step-cli You need the `step` binary from Smallstep: ```bash # Ubuntu/Debian wget https://dl.smallstep.com/gh-release/cli/docs-cli-install/v0.28.5/step-cli_0.28.5_amd64.deb && sudo dpkg -i step-cli_0.28.5_amd64.deb # macOS brew install step # Other platforms: https://smallstep.com/docs/step-cli/installation/ ``` ## Step 2 — Bootstrap against the CCAT CA (CCAT-ROOT-FINGERPRINT-CANONICAL) This downloads the CA's root certificate and saves it to `/home//.step/`: ```bash step ca bootstrap --ca-url https://ca.ccat.uni-koeln.de --fingerprint da7e917de3ed65a373872374ed981bc46fd81aed609d531ca133a3c1369f6255 ``` The fingerprint above is the **HSM-backed CCAT root** (Phase 2, established 2026-04-29). To verify against an independent source, check the ceremony record in the git log: ```bash git log --grep="ca_trust: commit" --oneline ``` Verify the bootstrap worked: ```bash step ca health ``` Should return `ok`. ## Step 3 — Issue your first certificate ```bash mkdir -p ~/.step/ssh step ssh certificate $(id -un) ~/.step/ssh/id_ccat --provisioner CCAT-GitHub --no-password --insecure --force ``` A browser tab opens, you authenticate with GitHub, Dex checks your team membership, and a 16-hour SSH certificate is written to disk. Three files appear: - `~/.step/ssh/id_ccat` — private key (unencrypted, short-lived) - `~/.step/ssh/id_ccat.pub` — public key - `~/.step/ssh/id_ccat-cert.pub` — the signed certificate (16h) ```{note} The private key is unencrypted on disk (`--no-password --insecure`). This is the right tradeoff: the cert expires in 16 hours, and an unencrypted key enables transparent renewal without password prompts. The real security boundary is the OIDC flow (GitHub team membership), not the key file. Your disk encryption (FDE) provides the at-rest protection layer. ``` ## Step 4 — Install the renewal helper The renewal helper is a 15-line script that checks cert freshness before each SSH connection and opens a browser for re-auth when the cert has expired. It lives in the system-integration repo: ```bash mkdir -p ~/.local/bin cp tools/step-ssh-renew-ccat ~/.local/bin/ chmod +x ~/.local/bin/step-ssh-renew-ccat ``` Make sure `~/.local/bin` is on your `PATH`. Most Ubuntu/Debian systems include it by default; if not, add to your shell rc: ```bash export PATH="$HOME/.local/bin:$PATH" ``` ## Step 5 — Configure SSH Add this to `~/.ssh/config`. The `Match exec` block runs the renewal helper before every CCAT connection; the `Host` block sets up the jump host, identity ordering, and fallback. **For operators WITH a Nitrokey (Tier 1 admins):** ``` # CCAT cert auto-renewal — opens browser if cert expired Match host "input-?,input-?-staging,*.data.ccat.uni-koeln.de,*.staging.data.ccat.uni-koeln.de" exec "step-ssh-renew-ccat" IdentityFile ~/.step/ssh/id_ccat CertificateFile ~/.step/ssh/id_ccat-cert.pub # CCAT hosts — cert first, Nitrokey fallback Host input-? input-?-staging *.data.ccat.uni-koeln.de *.staging.data.ccat.uni-koeln.de IdentityFile ~/.ssh/id_ed25519_sk_datacenter IdentityFile ~/.ssh/id_ed25519_sk_datacenter_backup IdentitiesOnly yes ``` **For operators WITHOUT a Nitrokey (Tier 2 staff):** ``` # CCAT cert auto-renewal — opens browser if cert expired Match host "input-?,input-?-staging,*.data.ccat.uni-koeln.de,*.staging.data.ccat.uni-koeln.de" exec "step-ssh-renew-ccat" IdentityFile ~/.step/ssh/id_ccat CertificateFile ~/.step/ssh/id_ccat-cert.pub # CCAT hosts — cert only (no hardware fallback) Host input-? input-?-staging *.data.ccat.uni-koeln.de *.staging.data.ccat.uni-koeln.de IdentitiesOnly yes ``` How the config works: 1. **`Match exec` fires first.** If the cert is valid, the `Match` block applies and its `IdentityFile` + `CertificateFile` are added to the identity list. If the cert is expired, the browser opens for renewal; if renewal succeeds, same result. If renewal fails (CA down), the `Match` block is skipped entirely. 2. **`Host` block always applies.** It adds `IdentitiesOnly yes` and for Tier 1 admins the Nitrokey `IdentityFile` entries. (User, ProxyJump, and HostName come from your own existing config blocks — this block handles identity/cert only.) 3. **OpenSSH offers identities in order:** cert first (from `Match`), then Nitrokeys (from `Host`). If the cert was accepted, done. If not (expired, CA down), Nitrokey takes over — the dongle blinks, you tap it, you're in. 4. **`IdentitiesOnly yes`** ensures only these 1-3 identities are offered. The other 10+ keys in your agent are never sent. No fail2ban triggers, no MaxAuthTries exhaustion. ## Step 6 — Test it ```bash ssh -v input-a 2>&1 | grep -iE 'offering|accepted|authent' ``` You should see the cert offered first and accepted. If the cert has expired, a browser tab opens first, you click authorize, then the connection proceeds. ## Renewal and expiry Certs expire after 16 hours. This is by design — see {doc}`background/ca-architecture` § "Why these lifetimes" for the rationale. You never need to think about it: - **Cert valid:** `ssh input-a` connects instantly. No browser, no prompt, no delay. - **Cert expired:** `ssh input-a` opens a browser tab for GitHub OAuth (~3 seconds), then connects. You see the browser — that's your awareness signal that the infrastructure is working. - **CA down (Tier 1 admin):** `ssh input-a` skips the cert, offers your Nitrokey instead. Dongle blinks. Report the CA failure to the team. - **CA down (Tier 2 staff):** `ssh input-a` fails with "Permission denied." Contact an admin. This is the intended failure mode — the CA being down is an incident, and Tier 2 staff not being able to SSH is how the team notices. ## Bootstrapping from off-network The CA vhost (`ca.ccat.uni-koeln.de`) is locked to Uni Köln `/16` at the proxy ({file}`proxy/data/vhost.d/ca.ccat.uni-koeln.de`). If you hit the bootstrap from a non-uni IP you'll see a TLS handshake that never completes (the proxy returned `403 Forbidden` before negotiating). There are two supported options: 1. **Ask a CCAT operator to add your CIDR to the allowlist.** Best choice for stable workstations. The operator follows *Adding a partner subnet* in {doc}`ca-day-to-day`. 2. **Tunnel the bootstrap through hera.** Hera is on Uni Köln, so traffic exiting hera reaches the proxy from an allowlisted IP. Open a local-forward in one shell: ```bash ssh -N -L 8443:ca.ccat.uni-koeln.de:443 hera.uni-koeln.de ``` Then in another shell, point step at the local end and pass the real SNI hostname so the proxy serves the CA vhost: ```bash step ca bootstrap \ --ca-url https://ca.ccat.uni-koeln.de:8443 \ --fingerprint da7e917de3ed65a373872374ed981bc46fd81aed609d531ca133a3c1369f6255 ``` Once the root is installed locally you can drop the tunnel; daily `ssh` works through hera as a normal `ProxyJump`. There is **no** client-side trust-bundle workaround. Off-uni clients are stopped at the proxy IP allowlist, before TLS ever negotiates, so no trust trick on your end can help. ## Optional: cert auth through hera Hera is a university jump host used as a TCP tunnel via `ProxyJump`. Cert auth on hera is **not required** — ProxyJump uses `-W` mode, which forwards the connection without authenticating to hera in any special way beyond your normal credentials. If you want cert auth on hera too (so the step-ca cert works for the jump as well), add the CCAT SSH user CA to your `authorized_keys` on hera: ```bash # Get the CA public key from the repo cat ansible/roles/ca_trust/files/ssh_user_ca.pub # On hera, append to your authorized_keys (replace ): echo 'cert-authority,principals="" ' >> ~/.ssh/authorized_keys ``` ```{warning} The `principals=""` restriction is **mandatory**. Without it, any cert signed by the CCAT user CA — for any principal — could log in as you on hera. Always restrict to your own principal. ``` ## Troubleshooting **`x509: certificate signed by unknown authority` after bootstrap** The CA vhost (`ca.ccat.uni-koeln.de`) is fronted by nginx-proxy with a CCAT-rooted cert. If you see this error from a uni-Köln IP the proxy is serving the wrong cert — verify what it's presenting: ```bash echo | openssl s_client -connect ca.ccat.uni-koeln.de:443 -servername ca.ccat.uni-koeln.de 2>/dev/null | openssl x509 -noout -issuer # Expect: issuer=O=CCAT Observatory, CN=CCAT Intermediate CA ``` If the issuer is `Let's Encrypt`, the renewal/re-issue runbook in {doc}`ca-rotation-and-recovery` § "Vhost cert emergency re-issue" needs to run on input-b. From off-uni you'll see the same TLS error because the proxy `403`s your request before negotiating — see *Bootstrapping from off-network* above. **Browser opens but Dex says "access denied"** You are not in the `ccatobs/datacenter` GitHub team. Ask a team maintainer to add you at `https://github.com/orgs/ccatobs/teams/datacenter/members`. **Cert issued but SSH says "Permission denied"** Check principals: `ssh-keygen -L -f ~/.step/ssh/id_ccat-cert.pub | grep Principals`. Your GitHub username must be listed. Then check the target host has `ca_trust` applied — specifically that `/etc/ssh/auth_principals/` exists and contains your GitHub username. **SSH works but offers Nitrokey first instead of cert** The `Match exec` block must appear BEFORE the `Host` block in your `~/.ssh/config`. OpenSSH applies blocks in order; if the `Host` block's `IdentityFile` entries are seen before the `Match` block's, they're offered first. **`step-ssh-renew-ccat: command not found`** The script is not on your `PATH`. Either move it to a directory that is (e.g. `/usr/local/bin/`), or add `~/.local/bin` to your `PATH` in your shell rc. For deeper troubleshooting (CA-side issues, provisioner errors, rotation procedures), see {doc}`ca-day-to-day` and {doc}`ca-provisioner-management`. ## See also - {doc}`ssh-access` — the short, operator-facing entry point (`ccat ssh setup`-driven). - {doc}`background/ca-architecture` — design context: why GitHub OIDC, what the lifetimes mean. - {doc}`background/certificate-authority-threat-model` — what cert auth defends against, and what it does not.