Operations Database (ops-db)
=============================

.. verified:: 2025-01-21
   :reviewer: Christof Buchbender

The ops-db package provides the core PostgreSQL database schema and SQLAlchemy ORM layer
for the CCAT Data Center operations database. This is the **single source of truth** for
all operational metadata: from observation planning through execution, data transfer,
archival, to final publication.

This documentation covers the **DATABASE SCHEMA ONLY** - not infrastructure deployment
(see :doc:`/system-integration/docs/index`) or the REST API layer (see
:doc:`/ops-db-api/docs/index`) or the web interface (see :doc:`/ops-db-ui/docs/index`).
:doc:`/docs/operations/index` explains how all these components work together and are
deployed and maintained.

The database serves as the backend for:

* **ops-db-ui** - Web interface for browsing observations and monitoring system state
* **ops-db-api** - RESTful API for instruments to register observations and data
* **data-transfer** - Automated workflows that move data through its lifecycle

What's in the Database
----------------------

The ops-db tracks:

* **Observatory Infrastructure** - Observatories, telescopes, instruments, and modules
* **Scientific Programs** - Observing programs, sub-programs, and observation units
* **Observation Execution** - Executed observation units with timing and conditions
* **Data Files and Packages** - Raw data files, packages, and physical copies across locations
* **Transfer Infrastructure** - Sites, locations, routes, and transfer records
* **Long-Term Archive** - Archive transfers and staging jobs for processing
* **Access Control** - Users, roles, and API tokens
* **Future Extensions** - Reduction pipelines and workflows, Scientific Data Products,
  Publications, etc.

For complete details on all models, see the :doc:`source/schema/table_reference` and
:doc:`source/api_reference/models` documentation.

Documentation Structure
-----------------------

.. grid:: 1 2 2 2
   :gutter: 2

   .. grid-item-card::
      :text-align: center
      :link: source/concepts/overview
      :link-type: doc

      **Concepts**
      ^^^^^^^^^^^^

      Understanding the database design and entity relationships

   .. grid-item-card::
      :text-align: center
      :link: source/schema/database_layout
      :link-type: doc

      **Schema Reference**
      ^^^^^^^^^^^^^^^^^^^

      Database diagram and auto-generated table documentation

   .. grid-item-card::
      :text-align: center
      :link: source/api_reference/core
      :link-type: doc

      **API Reference**
      ^^^^^^^^^^^^^^^^^

      Python API for working with the database

   .. grid-item-card::
      :text-align: center
      :link: source/config/index
      :link-type: doc

      **Configuration**
      ^^^^^^^^^^^^^^^^

      How to configure database connections and settings

   .. grid-item-card::
      :text-align: center
      :link: source/integration/related_components
      :link-type: doc

      **Integration**
      ^^^^^^^^^^^^^^^

      How this fits with other CCAT components

Quick Links
-----------

* :doc:`/ops-db-api/docs/index` - RESTful API for programmatic access
* :doc:`/ops-db-ui/docs/index` - Web interface for browsing and management
* :doc:`/data-transfer/docs/index` - Automated data transfer workflows
* :doc:`/system-integration/docs/index` - Deployment and infrastructure
* :doc:`/docs/source/operations/index` - Operations of the CCAT Data Center

Getting Started for Developers
-------------------------------

To use ops-db in your Python code:

1. Install the package::

      pip install -e /path/to/ops-db

2. Initialize a database connection::

      from ccat_ops_db import init_ccat_ops_db
      session, engine = init_ccat_ops_db()

3. Use the ORM models::

      from ccat_ops_db.models import ObsUnit, ExecutedObsUnit
      obs_unit = session.query(ObsUnit).filter_by(name="my_obs").first()

For detailed function documentation, see the :doc:`source/api_reference/core` section.

To set up a fresh database instance, use the ``opsdb_init`` command-line tool. See
:doc:`source/api_reference/utilities` for details.

Documentation Contents
----------------------

.. toctree::
   :maxdepth: 2
   :caption: Concepts:
   :hidden:

   source/concepts/overview
   source/config/index
   source/concepts/observatory_hierarchy
   source/concepts/observation_model
   source/concepts/data_model
   source/concepts/location_model
   source/concepts/transfer_model

.. toctree::
   :maxdepth: 2
   :caption: Schema:
   :hidden:

   source/schema/database_layout
   source/schema/table_reference

.. toctree::
   :maxdepth: 2
   :caption: API Reference:
   :hidden:

   source/api_reference/core
   source/api_reference/models
   source/api_reference/utilities

.. toctree::
   :maxdepth: 1
   :caption: Integration:
   :hidden:

   source/integration/related_components