Running Locally with Docker Compose
====================================

The fastest way to run the complete ops-db-api stack locally is using Docker Compose, which sets up the API, PostgreSQL, and Redis with a single command.

.. contents:: Table of Contents
   :local:
   :depth: 2

Prerequisites
-------------

* Docker (version 20.10 or higher)
* Docker Compose (version 2.0 or higher)

Verify you have Docker installed:

.. code-block:: bash

   docker --version
   docker compose version

Quick Start (Docker Compose)
-----------------------------

If a ``docker-compose.yml`` file exists in the repository:

.. code-block:: bash

   # Clone the repository
   git clone https://github.com/ccatobs/ops-db-api.git
   cd ops-db-api

   # Start all services
   docker compose up -d

   # View logs
   docker compose logs -f

   # Stop all services
   docker compose down

This will start:

* **API server** on port 8000
* **PostgreSQL** on port 5432
* **Redis** on port 6379

Access the API at http://localhost:8000/docs

Manual Docker Setup
-------------------

If Docker Compose configuration doesn't exist, you can run services manually:

Step 1: Create a Docker Network
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

   docker network create ccat-network

Step 2: Start PostgreSQL
~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

   docker run -d \
     --name ccat-postgres \
     --network ccat-network \
     -e POSTGRES_USER=ccat_ops_user \
     -e POSTGRES_PASSWORD=ccat_ops_password \
     -e POSTGRES_DB=ccat_ops_db \
     -p 5432:5432 \
     -v ccat-postgres-data:/var/lib/postgresql/data \
     postgres:14

Step 3: Start Redis
~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

   docker run -d \
     --name ccat-redis \
     --network ccat-network \
     -p 6379:6379 \
     -v ccat-redis-data:/data \
     redis:7

Step 4: Build and Run the API
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

   # Build the Docker image
   docker build -t ccat-ops-db-api .

   # Run the API container
   docker run -d \
     --name ccat-api \
     --network ccat-network \
     -p 8000:8000 \
     -e MAIN_DB_HOST=ccat-postgres \
     -e LOCAL_DB_HOST=ccat-postgres \
     -e REDIS_HOST=ccat-redis \
     -e SECRET_KEY=your-secret-key-change-in-production \
     ccat-ops-db-api

Connecting to Local vs Remote Database
---------------------------------------

The API supports connecting to different databases for main (write) and local (read) operations:

Development Configuration (Single Local DB)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For local development, use the same database for both:

.. code-block:: bash

   # .env file
   SITE_TYPE=main
   MAIN_DB_HOST=localhost
   MAIN_DB_PORT=5432
   LOCAL_DB_HOST=localhost
   LOCAL_DB_PORT=5432

This configuration:

* Writes directly to the database (no buffering)
* Reads from the same database
* Simplest setup for development

Observatory Simulation (Buffered Writes)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To test transaction buffering locally:

.. code-block:: bash

   # .env file
   SITE_TYPE=secondary
   MAIN_DB_HOST=remote-main-db.example.com  # Simulated remote
   MAIN_DB_PORT=5432
   LOCAL_DB_HOST=localhost  # Local replica
   LOCAL_DB_PORT=5432

This configuration:

* Buffers write operations in Redis
* Reads from local database
* Merges buffered data with database reads
* Simulates observatory environment

Testing Both Configurations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can switch between configurations by changing the ``SITE_TYPE`` environment variable:

.. code-block:: bash

   # Test as main site (direct writes)
   export SITE_TYPE=main
   uvicorn ccat_ops_db_api.main:app --reload --port 8000

   # Test as secondary site (buffered writes)
   export SITE_TYPE=secondary
   uvicorn ccat_ops_db_api.main:app --reload --port 8001

Setting Site Type for Testing
------------------------------

The ``SITE_TYPE`` environment variable controls the API's behavior:

SITE_TYPE=main
~~~~~~~~~~~~~~

**Behavior**:

* All write operations execute directly on the database
* No transaction buffering
* No LSN tracking
* Simplest configuration

**Use when**:

* Developing new endpoints
* Testing database queries
* Running at the main site (Cologne)

SITE_TYPE=secondary
~~~~~~~~~~~~~~~~~~~

**Behavior**:

* Critical operations buffered in Redis
* Background processor executes buffered transactions
* LSN tracking monitors replication
* Smart queries merge database + buffered data

**Use when**:

* Testing transaction buffering
* Simulating observatory environment
* Testing replication lag scenarios

Example .env for Development
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

   # Development configuration (main site)
   SITE_NAME=development
   SITE_TYPE=main
   MAIN_DB_HOST=localhost
   LOCAL_DB_HOST=localhost
   REDIS_HOST=localhost

Example .env for Observatory Simulation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

   # Observatory simulation (secondary site)
   SITE_NAME=observatory
   SITE_TYPE=secondary
   MAIN_DB_HOST=main-db-host  # Points to main DB
   LOCAL_DB_HOST=localhost     # Local replica
   REDIS_HOST=localhost
   CRITICAL_OPERATIONS_BUFFER=true
   LSN_TRACKING_ENABLED=true

Accessing Logs and Debugging
-----------------------------

Docker Compose Logs
~~~~~~~~~~~~~~~~~~~

View logs from all services:

.. code-block:: bash

   # All services
   docker compose logs -f

   # Specific service
   docker compose logs -f api

   # Last 100 lines
   docker compose logs --tail=100 api

Individual Container Logs
~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

   # API logs
   docker logs -f ccat-api

   # PostgreSQL logs
   docker logs -f ccat-postgres

   # Redis logs
   docker logs -f ccat-redis

Accessing the Database
~~~~~~~~~~~~~~~~~~~~~~

Connect to PostgreSQL using psql:

.. code-block:: bash

   # With Docker Compose
   docker compose exec postgres psql -U ccat_ops_user -d ccat_ops_db

   # With individual container
   docker exec -it ccat-postgres psql -U ccat_ops_user -d ccat_ops_db

.. Example queries:

.. .. code-block:: sql

..    -- List all tables
..    \dt

..    -- View executed observation units
..    SELECT * FROM executed_obs_unit LIMIT 10;

..    -- Check transaction buffer size
..    SELECT COUNT(*) FROM executed_obs_unit WHERE created_at > NOW() - INTERVAL '1 hour';

Accessing Redis
~~~~~~~~~~~~~~~

Connect to Redis using redis-cli:

.. code-block:: bash

   # With Docker Compose
   docker compose exec redis redis-cli

   # With individual container
   docker exec -it ccat-redis redis-cli

Example commands:

.. code-block:: 

   # Check connection
   PING

   # List all keys
   KEYS *

   # View transaction buffer
   LRANGE site:development:transaction_buffer 0 -1

   # Check buffer size
   LLEN site:development:transaction_buffer

See :doc:`../development/redis-inspection` for more Redis debugging commands.

API Logs in Development Mode
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When running with uvicorn directly (not Docker):

.. code-block:: bash

   # Start with INFO level logging
   uvicorn ccat_ops_db_api.main:app --reload --log-level info

   # Debug level for detailed logs
   uvicorn ccat_ops_db_api.main:app --reload --log-level debug

Logs will show:

* Request/response for each API call
* Database queries executed
* Transaction buffering operations
* Background processor activity
* LSN tracking progress

Hot Reload for Development
---------------------------

When running locally (not in Docker), uvicorn's ``--reload`` flag enables hot reload:

.. code-block:: bash

   uvicorn ccat_ops_db_api.main:app --reload --port 8000

This will automatically restart the server when you modify:

* Python files in ``ccat_ops_db_api/``
* Router files
* Schema definitions
* Configuration files

.. note::
   Hot reload only works when running uvicorn directly, not in Docker containers. For Docker development, mount your code as a volume.

Stopping and Cleaning Up
-------------------------

Docker Compose
~~~~~~~~~~~~~~

.. code-block:: bash

   # Stop services (keeps data)
   docker compose stop

   # Stop and remove containers (keeps volumes)
   docker compose down

   # Remove everything including volumes (⚠️ deletes data)
   docker compose down -v

Individual Containers
~~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

   # Stop containers
   docker stop ccat-api ccat-postgres ccat-redis

   # Remove containers
   docker rm ccat-api ccat-postgres ccat-redis

   # Remove volumes (⚠️ deletes data)
   docker volume rm ccat-postgres-data ccat-redis-data

Common Issues and Solutions
----------------------------

Port Already in Use
~~~~~~~~~~~~~~~~~~~

If port 8000, 5432, or 6379 is already in use:

.. code-block:: bash

   # Find process using port
   lsof -i :8000

   # Kill the process
   kill -9 <PID>

   # Or use different ports
   docker run -p 8001:8000 ...

Database Connection Refused
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If the API can't connect to the database:

1. Verify PostgreSQL container is running: ``docker ps``
2. Check network connectivity: ``docker network inspect ccat-network``
3. Verify environment variables match database credentials
4. Check database logs: ``docker logs ccat-postgres``

Redis Connection Errors
~~~~~~~~~~~~~~~~~~~~~~~

If Redis connection fails:

1. Verify Redis container is running: ``docker ps``
2. Test connection: ``docker exec ccat-redis redis-cli ping``
3. Check ``REDIS_HOST`` environment variable
4. Verify Redis is accessible: ``telnet localhost 6379``

Container Won't Start
~~~~~~~~~~~~~~~~~~~~~

If containers fail to start:

1. Check logs: ``docker logs <container-name>``
2. Verify the image exists: ``docker images``
3. Check for port conflicts: ``docker ps -a``
4. Remove and recreate: ``docker rm <container> && docker run ...``

Next Steps
----------

Now that you have the API running locally:

* :doc:`first-api-call` - Make your first authenticated request
* :doc:`../development/debugging-buffering` - Learn to debug transaction buffering
* :doc:`../tutorials/simple-endpoints/simple-read-endpoint` - Create your first endpoint
* :doc:`../architecture/site-configuration` - Understand site configuration in depth