Skip to content

Conversation

@joncarter1
Copy link

@joncarter1 joncarter1 commented Dec 19, 2025

What does this PR do?

Reinstates the optional checkpoint_path_prefix argument for the MLFlowLogger.

This was added in lightning==2.5.1 but removed in lightning==2.6.0.

Why?

lightning<2.5 logged model checkpoints in MLFlow under a hard-coded subfolder: model/checkpoints
lightning==2.5.0 changed this to log checkpoints under the root folder within a run.
This location change broke the MLOps tooling of myself and others who programatically retrieved these checkpoints.

The checkpoint_path_prefix argument added by @benglewis in #20538 meant that the previous behaviour could be restored. It was susceptible to a bug in MLFlow, so a patch in Lightning was introduced in #20669

However, both changes were yanked from the master branch:

The root issue was later fixed in MLFlow in mlflow/mlflow#16660 (MLFlow 3.6)

Closes #20932

Before submitting
  • Was this discussed/agreed via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist
  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

📚 Documentation preview 📚: https://pytorch-lightning--21432.org.readthedocs.build/en/21432/

Signed-off-by: Jonathan Carter <jonathan.carter.1@outlook.com>
@github-actions github-actions bot added docs Documentation related pl Generic label for PyTorch Lightning package labels Dec 19, 2025
@joncarter1 joncarter1 changed the title Reinstate checkpoint_path_prefix Reinstate checkpoint_path_prefix in MLFLowLogger Dec 19, 2025
@joncarter1 joncarter1 changed the title Reinstate checkpoint_path_prefix in MLFLowLogger Reinstate checkpoint_path_prefix in MLFlowLogger Dec 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation related has conflicts pl Generic label for PyTorch Lightning package

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MlflowException when logging checkpoints with MLFlowLogger

1 participant