TpT

[MathiasValla]

Hi Simon,

I’m happy to share that the TpT implementation is now finalized code-wise on my side. This MR introduces a working TpTDecisionTreeClassifier with a depth-first builder, compatible with scikit-lexicographical-trees / scikit-longitudinal.

References to cite

[1] Valla, M. Time-penalised trees (TpT): introducing a new tree-based data mining algorithm for time-varying covariates. Ann Math Artif Intell 92, 1609–1661 (2024). https://doi.org/10.1007/s10472-024-09950-w

[2] Mathias Valla, Xavier Milhaud. Time-penalized trees: consistency results and simulations. 2025. ⟨hal-05022929⟩ https://cnrs.hal.science/hal-05022929

Scope of this PR (minimal, functional)

• Estimator: TpTDecisionTreeClassifier (classification only in this PR’s target scope).
• Builder: DepthFirstTreeBuilder (no Best-First in this scope).
• Criterion: Gini (others left out intentionally for the minimal version).
• I/O and structure: Cython implementation aligned with the existing tree stack.
• (No plot_tree adaptation included here; regular sklearn.tree.plot_tree is usable for quick inspection.)

Note: I also have a working regression path and more features locally, but to keep this PR focused and easy to land, I’m proposing only the minimal, agreed-upon surface.

Where the code lives

• TpT estimator code: scikit_longitudinal/estimators/trees/TpT/
  (primary class: TpTDecisionTreeClassifier, splitter, builder, structs)
• Long→Wide converter (temporary): scikit_longitudinal/estimators/trees/TpT/_preprocessing.py
  This should be moved under or near LongitudinalDataset per your design. I’d be grateful if you could drop it into the right place in a temporary branch; I’ll review once moved. It currently handles long → wide only (not the reverse), without TIDAL/polars. It’s a first step toward 💡 Feature Request - From Wide to Long and vice-versa Longitudinal data formatting, inspired from TIDAL #64 but does not fully solve it.

Quick example (concise)

import pandas as pd
from sklearn import tree
from scikit_longitudinal.estimators.trees import TpTDecisionTreeClassifier

# Example dataset from tests (long format):
# columns include: ["id", "time_point", "duration", "target", ... features ...]
df = pd.read_csv("path/to/stroke.csv", sep=";")

X = df.drop(columns=["target", "cholesterol"])  # minimal cleanup for this file
y = df["target"].astype(int)

clf = TpTDecisionTreeClassifier(
    gamma=0.1,              # time penalty (λ)
    criterion="gini",
    id_col="id",
    time_col="time_point",
    duration_col="duration",
    assume_long_format=True,
    time_step=1,
    max_horizon=1000,
    min_samples_split=2,
    max_depth=1000,
    random_state=42,
).fit(X, y)

# Optional: quick textual sanity checks
print("max_depth:", clf.tree_.max_depth)
print("node_count:", clf.tree_.node_count)
print("split_time_index[:10]:", clf.tree_.split_time_index[:10])

# Optional: quick visual check (works for fast inspection)
feature_names = getattr(clf, "_wide_feature_names_", None)
_ = tree.plot_tree(
    clf, filled=True, fontsize=10,
    feature_names=feature_names, tpt_time_scale=0.5
)

That’s intentionally minimal (no non-essential utilities, no external metrics). It just loads data, fits TpT, prints a few structural fields, and plots.

Dependencies

• I temporarily removed ray from the dependencies due to local issues.
  Please feel free to restore it where appropriate (e.g., as an optional dependency / extra for parallelism or tests).

Ask / next steps

I’ve pushed this as far as I can right now. It would be ideal if you could manage the integration so TpT aligns perfectly with sklong’s patterns (API surface, dataset plumbing via LongitudinalDataset, docs structure, examples, CI, etc.). I’ll follow up with any fixes you need during review.

I’m also working on two additional papers involving TpT and will of course cite scikit-longitudinal as the reference implementation. For any future article specifically about the implementation, I’d be happy to include you as co-author for your guidance and help.

Thanks a lot, and please let me know how you’d like to proceed with the preprocessing relocation!

— Mathias