Read Buffer Manager
===================

The Read Buffer Manager tracks mutable updates to buffered records, enabling modifications before replication completes.

.. contents:: Table of Contents
   :local:
   :depth: 2

The Problem
-----------

**Scenario**: Observation buffered at T0, needs update at T1 (before replication).

.. code-block:: python

   # T0: Start observation (buffered)
   obs_id = await start_observation({"status": "running"})
   
   # T1: Observation finishes (but still in buffer!)
   await finish_observation(obs_id, {"status": "completed", "end_time": "..."})

**Can't update database** (record not there yet)

**Can't modify buffer** (breaks transaction atomicity)

The Solution
------------

**Read buffer**: Separate cache tracking updates to buffered records.

Core Implementation
-------------------

Key code from ``read_buffer_manager.py``:

.. literalinclude:: ../../../ccat_ops_db_api/transaction_buffering/read_buffer_manager.py
   :language: python
   :lines: 19-80

How It Works
------------

**Update read buffer**:

.. code-block:: python

   await read_buffer_manager.update_read_buffer(
       table_name="ExecutedObsUnit",
       record_id=obs_id,
       updates={"status": "completed", "end_time": "2025-01-01T01:00:00Z"},
       transaction_id=original_tx_id
   )

**Smart query merges**:

1. Query database (may be empty)
2. Query buffered cache (original record)
3. Query read buffer (updates)
4. **Merge**: Apply read buffer updates to buffered record

**Cleanup** when LSN confirms replication.

Use Cases
---------

1. **Finish buffered observation**
2. **Update buffered data file metadata**
3. **Modify buffered package status**

Redis Structure
---------------

.. code-block:: text

   Key: read_buffer:{table}:{id}
   Type: Hash
   Fields: Updated field values
   TTL: Extended until replicated

Update History
--------------

.. code-block:: text

   Key: read_buffer:{table}:{id}:updates
   Type: List
   Content: Timestamped update history

Transaction Affinity
--------------------

.. code-block:: text

   Key: read_buffer:transaction:{tx_id}:affected_records
   Type: Set
   Content: List of records affected by transaction

Cleanup triggers when LSN confirms replication of original transaction.

Next Steps
----------

* :doc:`smart-query-manager` - How read buffer is merged
* :doc:`../../tutorials/complex-endpoints/buffered-critical-operations` - Using in endpoints