Deep Dive
=========

This section provides detailed technical documentation on the core systems and implementations within ops-db-api.

.. contents:: Table of Contents
   :local:
   :depth: 2

Overview
--------

The deep dive documentation covers:

* **Transaction Buffering**: Complete buffering system architecture
* **Authentication**: Unified auth implementation details
* **Routers**: Endpoint organization and implementation
* **Data Flow**: Complete request/response patterns
* **Caching Strategy**: Redis caching patterns and policies

These guides are intended for developers who need to:

* Understand implementation details
* Extend the system with new features
* Debug complex issues
* Optimize performance

Deep Dive Sections
------------------

.. toctree::
   :maxdepth: 2

   transaction-buffering/index
   authentication/index
   routers/index
   data-flow
   caching-strategy

Transaction Buffering
~~~~~~~~~~~~~~~~~~~~~

The transaction buffering system is the core innovation that enables reliable operations despite network issues.

Topics covered:

* System lifecycle and architecture
* Transaction builder and dependencies
* Redis buffering and queue management
* Background processing patterns
* LSN-based replication tracking
* Smart query merging
* Read buffer for mutable updates

See :doc:`transaction-buffering/index` for details.

Authentication
~~~~~~~~~~~~~~

Unified authentication supporting both interactive users and automation.

Topics covered:

* Unified authentication interface
* GitHub OAuth flow implementation
* API token generation and management
* Role-based access control
* Permission system

See :doc:`authentication/index` for details.

Routers
~~~~~~~

Endpoint organization, categorization, and implementation patterns.

Topics covered:

* UI-focused routers
* Operations-focused routers
* Shared routers
* Request/response patterns

See :doc:`routers/index` for details.

Data Flow
~~~~~~~~~

Complete data flow examples from request to response.

See :doc:`data-flow` for examples.

Caching Strategy
~~~~~~~~~~~~~~~~

Redis caching patterns, TTL strategies, and invalidation.

See :doc:`caching-strategy` for details.

Prerequisites
-------------

Before diving into these topics, you should be familiar with:

* FastAPI framework basics
* SQLAlchemy ORM
* Redis data structures
* PostgreSQL replication concepts
* Async Python (asyncio)

Recommended reading order:

1. **Start with philosophy**: :doc:`../philosophy/index` - Understand the "why"
2. **Then architecture**: :doc:`../architecture/index` - Understand the "what"
3. **Then deep dives**: This section - Understand the "how"
4. **Finally tutorials**: :doc:`../tutorials/index` - Apply the knowledge

Code References
---------------

All deep dive documentation includes extensive code references using ``literalinclude`` directives that point to actual implementation files.

**Key source directories**:

* ``ccat_ops_db_api/transaction_buffering/`` - Buffering system
* ``ccat_ops_db_api/auth/`` - Authentication
* ``ccat_ops_db_api/routers/`` - API endpoints
* ``ccat_ops_db_api/dependencies.py`` - Dependency injection
* ``ccat_ops_db_api/main.py`` - Application setup

Conventions
-----------

Code Examples
~~~~~~~~~~~~~

Code examples are shown in two ways:

1. **Inline examples** - Simplified, pedagogical code
2. **literalinclude** - Actual production code with line references

.. code-block:: python

   # Inline example (simplified)
   def simple_example():
       return "hello"

.. literalinclude:: ../../ccat_ops_db_api/main.py
   :language: python
   :lines: 1-10


Diagrams
~~~~~~~~

**Mermaid diagrams** for:

* Sequence diagrams (request flows)
* Architecture diagrams (system components)
* State machines (transaction lifecycle)

**ASCII diagrams** for:

* Simple flows
* Tree structures

Cross-References
~~~~~~~~~~~~~~~~

Extensive cross-references connect related topics:

* ``:doc:`` - Links to other documentation pages
* ``:ref:`` - Links to specific sections
* ``:meth:`` - Links to methods (when using Sphinx autodoc)

Testing Deep Dive Concepts
---------------------------

Each deep dive topic includes suggestions for hands-on testing:

**Transaction Buffering**:

.. code-block:: bash

   # Set as secondary site
   export SITE_TYPE=secondary
   
   # Monitor buffer
   watch -n 1 curl -s http://localhost:8000/buffer-stats

**Authentication**:

.. code-block:: bash

   # Test JWT
   curl -H "Authorization: Bearer $JWT_TOKEN" http://localhost:8000/auth/me
   
   # Test API token
   curl -H "Authorization: Bearer $API_TOKEN" http://localhost:8000/auth/me

**Caching**:

.. code-block:: bash

   # Monitor Redis
   redis-cli MONITOR

Debugging Tips
--------------

Enable verbose logging:

.. code-block:: bash

   export LOG_LEVEL=DEBUG
   uvicorn ccat_ops_db_api.main:app --reload --log-level debug

Use API health endpoints:

.. code-block:: bash

   curl http://localhost:8000/health
   curl http://localhost:8000/buffer-stats
   curl http://localhost:8000/api/site/info

Inspect Redis directly:

.. code-block:: bash

   redis-cli
   > LLEN site:observatory:transaction_buffer
   > LRANGE site:observatory:transaction_buffer 0 -1

Next Steps
----------

Choose a deep dive topic:

* :doc:`transaction-buffering/index` - Start with the buffering system
* :doc:`authentication/index` - Learn authentication implementation
* :doc:`routers/index` - Understand endpoint organization
* :doc:`data-flow` - See complete request/response flows
* :doc:`caching-strategy` - Optimize with caching