TDP Lib

tdp-lib is a Python library built on top of Ansible. It is designed for managing TDP clusters, offering advanced tools to extend Ansible's capabilities. Key features include:

• Creating a Directed Acyclic Graph (DAG) of tasks to manage dependencies and relationships between services and components.
• Centralized configuration management through a unified variable definition system.

tdp-lib can be utilized in various ways: as a Python library, through an admin CLI, via a REST API (see tdp-server), or through a web interface (see tdp-ui).

Pre-requisites

To use tdp-lib, ensure you have the following prerequisites:

• Python 3.9 or higher. Higher version are not guarantee to work, only 3.9 is tested.
• A relational database management system (RDBMS), such as PostgreSQL or SQLite.

Optional dependencies for DAG visualization:

• Graphviz for graphical representation of DAGs.

Installation

Set the following environment variables:

• TDP_COLLECTION_PATH: Specifies the file path(s) to the necessary collection(s). tdp-collection is mandatory. Multiple collections can be specified, separated by a colon : (e.g., tdp-collection-extras, tdp-observability).
• TDP_RUN_DIRECTORY: Path to the working directory of the TDP deployment (where ansible.cfg, inventory.ini, and topology.ini are located).
• TDP_DATABASE_DSN: Database DSN (Data Source Name) for the chosen RDBMS.
• TDP_VARS: Path to the folder containing configuration variables.

Ensure Ansible is configured to use the tosit.tdp.inventory plugin. Example ansible.cfg:

[defaults]
inventory=your_inventory,..,~/tdp_vars

[inventory]
enable_plugins = tosit.tdp.inventory,..,your_plugins

Install the library:

# Create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate
# Install the dependencies
pip install "tdp-lib[visualization]@https://github.com/TOSIT-IO/tdp-lib/tarball/1.0.0"
# Initialize the database and tdp_vars
tdp init

Supported databases

tdp-lib uses SQLite by default. Other backends are supported:

• PostgreSQL - install with pip install tdp-lib[postgresql]
• MySQL/MariaDB - install with pip install tdp-lib[mysql]

CLI Usage

Note

This section is a work in progress.

tdp --help

Contributing

Contributions are welcome! Here are some guidelines specific to this project:

Dev environment

Use uv for development:

# Install dependencies
uv sync --all-extras

Testing

Tests are based on Pytest:

uv run pytest

tdp-lib is supporting SQLite, PostgreSQL and MariaDB. A compose.yaml is provided to run test accross all suported environments:

docker compose -f dev/docker-compose.yaml up -d
uv run pytest tests --database-dsn 'postgresql+psycopg2://postgres:postgres@localhost:5432/tdp' --database-dsn 'mysql+pymysql://mysql:mysql@localhost:3306/tdp' --database-dsn 'mysql+pymysql://mariadb:mariadb@localhost:3307/tdp'
docker compose -f dev/docker-compose.yaml down -v

Contributions

Commit messages must adhere to the Conventional Commits standard.

Code must be formated and ensure the linting rules before being merged. This project use Ruff:

# Format the code and reorder imports
uv run ruff check --select I --fix && uv run ruff format
# Check the code for linting issues
uv run ruff check
# Lint the code
uv run ruff check --fix