Merge pull request #66 from rmnldwg/release-1.0.0.a4

Release 1.0.0.a4

Author: rmnldwg

.pre-commit-config.yaml

@@ -10,11 +10,6 @@ repos:
     - id: ruff
       args: [ --fix ]
     - id: ruff-format
-- repo: https://github.com/asottile/pyupgrade
-  rev: v3.16.0
-  hooks:
-  - id: pyupgrade
-    args: [--py310-plus]
 - repo: https://github.com/compilerla/conventional-pre-commit
   rev: v3.3.0
   hooks:

CHANGELOG.md

@@ -2,6 +2,82 @@
 
 All notable changes to this project will be documented in this file.
 
+## [1.0.0.a4] - 2025-01-23
+
+### Bug Fixes
+
+- Update data loading to new lydata API.
+- Add sampling config back to sample CLI.
+- Finish `data filter` command.
+- Correctly log number of excluded pateints in `lyproxify`.
+- Allow extra args in CLI cmds.
+- Logging during progress bar.
+
+### Documentation
+
+- Deactivate help of removed commands.
+- Link only to stable versions.
+- Fix intersphinx links.
+- Update link to schedule module.
+- Configure how pydantic models are displayed.
+- Add more info about schema.
+- Better explain sampling.
+- Add proper info to `cli_cmd()` methods.
+
+### Features
+
+- Add mandatory `version` field to command settings.\
+  This will allow to differentiate between old and new configs and create
+  the models accordingly.
+- Add translation of old to new model configs.
+- Add dynamic YAML config source.
+- Configure logging nicely.
+- Update `data enhance` command.
+- Update `data join` command.
+- Update `data filter` command.
+- Update `data split` command.
+- Capture lydata logging output.
+- Update YAML schema for CLIs.
+- Update `data lyproxify` cmd.
+- Allow sampling only specified params.\
+  Via a new CLI arg named `param_names` one may restrict the parameters
+  sampled to a named subset. In combination with the fact that any Python
+  model may be loaded, this results in an enormously flexible sampler.
+- Update inv temp `schedule` cmd.
+- Allow providing start state to sampling func.
+
+### Testing
+
+- Replace subprocess calls with monkeypatch.\
+  This allows for better debugging during test calls.
+- Load generated data correctly.\
+  The synthetic data for testing already has "early" and "late" as
+  T-stages. Thus, the mapping needed to be adapted.
+
+### Build
+
+- pydantic-settings >= 2.7 needed.
+
+### Change
+
+- Make `version` field in command settings required.
+- Use pydantic for subcommands.
+- Use loguru over default logging.
+- Rename `data` field to `input`.
+- Use pydantic for plot utils, too.
+- Use rich logging handler.
+
+### Refac
+
+- Slightly change CLI inheritances.
+- Sort configs alphabetically.
+- Make sampling more reusable.
+
+### Remove
+
+- Unused utility functions.
+- Plotting scripts except histogram/betaposterp helpers.
+
 ## [1.0.0.a3] - 2024-11-15
 
 ### Bug Fixes

README.md

@@ -8,9 +8,9 @@
 
 ## What are these `lyscripts`?
 
-This package provides convenient scripts for performing inference and learning regarding the lymphatic spread of head & neck cancer. Essentially, it provides a *command line interface* (CLI) to the [lymph](https://github.com/rmnldwg/lymph) library.
+This package provides convenient scripts for performing inference and learning regarding the lymphatic spread of head & neck cancer. Essentially, it provides a *command line interface* (CLI) to the [lymph](https://github.com/rmnldwg/lymph) library and the [lydata](https://github.com/rmnldwg/lydata) repository that stores lymphatic progression data.
 
-We are making these "convenience" scripts public, because doing so is one necessary requirement to making our research easily and fully reproducible. There exists another repository, [lynference](https://github.com/rmnldwg/lynference), where we store the pipelines that produce(d) our published results in a persistent way. Head over there to learn more about how to reproduce our work.
+We are making these "convenience" scripts public, because doing so is one necessary requirement to making our research easily and fully reproducible. There exists another repository, [lynference](https://github.com/rmnldwg/lynference), where we stored the pipelines that produced our published results in a persistent way.
 
 ## Installation
 
@@ -32,7 +32,7 @@ pip install .
 
 This package is intended to be mainly used as a collection of Python scripts that would be scattered throughout my projects, if I didn't bundle them here. Hence, they're mostly command line tools that do basic and repetitive stuff.
 
-### As a command line tool
+### As a Command Line Tool
 
 Simply run
 
@@ -44,8 +44,24 @@ in your terminal and let the output guide you through the functions of the progr
 
 You can also refer to the [documentation] for a written-down version of all these help texts and even more context on how and why to use the provided commands.
 
-### As a library
+### As a Library
 
 Head over to the [documentation] for more information on the individual modules, classes, and functions that are implemented in this package.
 
 [documentation]: https://lyscripts.readthedocs.io
+
+### Configuration YAML Schema
+
+Most of the CLI commands allow passing a list of `--configs` in the form of YAML files. If for a particular CLI argument no value is passed directly, the program looks for the corresponding value in the merged YAML files (if multiple files are provided, later ones may overwrite earlier ones).
+
+For these YAML files we provide a unified schema containing all possible fields that any of the CLIs may accept. It is located at `schemas/ly.json` in this repository. So, one could configure e.g. VS Code to consider this schema for all `*.ly.yaml` files. Here is how that could look like in the JSON settings of VS Code:
+
+```json
+{
+    "yaml.schemas": {
+        "https://raw.githubusercontent.com/rmnldwg/lyscripts/main/schemas/ly.json": "*.ly.yaml"
+    }
+}
+```
+
+Subsequently, all files ending in `.ly.yaml` will have helpful autocompletion on the allowed/expected types available.

docs/source/compute/init.rst

@@ -1,7 +1,7 @@
 .. currentmodule:: lyscripts.compute
 
-Precomputing Quantitites
-========================
+Model Predictions
+=================
 
 .. automodule:: lyscripts.compute
     :members:

docs/source/compute/posteriors.rst

@@ -5,6 +5,7 @@ Posterior State Distributions
 
 .. automodule:: lyscripts.compute.posteriors
     :members:
+    :show-inheritance:
 
 Command Help
 ------------

docs/source/compute/prevalences.rst

@@ -5,6 +5,7 @@ Predict Prevalence of Involvement
 
 .. automodule:: lyscripts.compute.prevalences
     :members:
+    :show-inheritance:
 
 Command Help
 ------------

docs/source/compute/priors.rst

@@ -5,6 +5,7 @@ Prior State Distributions
 
 .. automodule:: lyscripts.compute.priors
     :members:
+    :show-inheritance:
 
 Command Help
 ------------

docs/source/compute/risks.rst

@@ -1,10 +1,11 @@
 .. currentmodule:: lyscripts.compute.risks
 
-Predict Risk of Involvement
+Compute Risk of Involvement
 ===========================
 
 .. automodule:: lyscripts.compute.risks
     :members:
+    :show-inheritance:
 
 Command Help
 ------------

docs/source/compute/utils.rst

@@ -5,3 +5,4 @@ Helpers for Computing Quantities
 
 .. automodule:: lyscripts.compute.utils
     :members:
+    :show-inheritance:

docs/source/conf.py

@@ -48,12 +48,24 @@
 # create links to other projects
 intersphinx_mapping = {
     "python": ("https://docs.python.org/3.10", None),
-    "lymph": ("https://lymph-model.readthedocs.io/latest/", None),
+    "lymph": ("https://lymph-model.readthedocs.io/stable/", None),
     "pandas": ("https://pandas.pydata.org/pandas-docs/stable/", None),
     "numpy": ("https://numpy.org/doc/stable/", None),
-    "lydata": ("https://lydata.readthedocs.io/latest/", None),
+    "lydata": ("https://lydata.readthedocs.io/stable/", None),
+    "emcee": ("https://emcee.readthedocs.io/en/stable/", None),
 }
 
+# autodoc_pydantic settings
+autodoc_pydantic_model_show_config_summary = False
+autodoc_pydantic_model_show_validator_members = False
+autodoc_pydantic_model_show_field_summary = False
+autodoc_pydantic_model_member_order = "bysource"
+autodoc_pydantic_settings_show_config_summary = False
+autodoc_pydantic_settings_show_validator_members = False
+autodoc_pydantic_settings_show_field_summary = False
+autodoc_pydantic_settings_member_order = "bysource"
+autodoc_pydantic_field_show_constraints = False
+
 
 # -- Options for HTML output -------------------------------------------------