New package to extract latents from high-dimensional data using LDS and TorchSharp #70

ncguilbeault · 2025-09-29T11:00:50Z

Summary

This PR introduces a new package, Bonsai.ML.Torch.LDS, which implements linear dynamical systems (LDS) using TorchSharp for online filtering, smoothing, and parameter estimation in Bonsai. This package differs from the Bonsai.ML.LinearDynamicalSystems package in that it does not depend on Python, and the models parameters can be customised. A test was added to compare the output of the TorchSharp implementation with the output of the lds_python package following parameter estimation and filtering of neural recordings.

The PR in #77 should be merged first.

Refactoring the unit test to download the expected results instead of generating them with a Python script fixes #75

glopesdev · 2025-10-10T11:42:36Z

Leaving here a summary of what we discussed earlier about deprecating the Bonsai.ML.LinearDynamicalSystems package and adopting back-end suffixes for systems with different implementations. In this case the steps would be:

Rename the package to Bonsai.ML.Lds.Torch
Rename Bonsai.ML.LinearDynamicalSystems to Bonsai.ML.Lds.Python
Deprecate the Bonsai.ML.LinearDynamicalSystems package

The one consideration we discussed was whether to "nest" namespaces, e.g. we could use the name Bonsai.ML.Torch.Lds to make it so that all types in Bonsai.ML.Torch would be automatically visible to ML packages using the Torch backend. In the end we decided to go with the suffix, and import functionality where needed, similar to what is done with most non-overlapping .NET namespaces.

ncguilbeault · 2025-10-13T17:28:05Z

Updating the namespace of the existing Bonsai.ML.LinearDynamicalSystems resulted in quite a few changes. Because of this, I decided to push these changes to a separate PR which we should merge first before this one. The new PR is here: #74

glopesdev

Looks good, left only a few minor comments to review.

glopesdev · 2025-10-24T16:09:36Z

docs/README.md

 > [!NOTE]
 > Bonsai.ML packages can be installed through Bonsai's integrated package manager and are generally ready for immediate use. However, some packages may require additional installation steps. Refer to the specific package section for detailed installation guides and documentation.

+### Bonsai.ML.Torch.LDS


Rename to match package and project name.

glopesdev · 2025-10-24T16:09:51Z

docs/articles/Torch.LDS/torch-lds-overview.md

@@ -0,0 +1,7 @@
+# Bonsai.ML.Torch.LDS - Overview


Same as above.

glopesdev · 2025-10-24T16:10:40Z

docs/articles/Torch.LDS/torch-lds-overview.md

+
+## Installation Guide
+
+Install the `Bonsai.ML.Torch.LDS` package from the Bonsai package manager. You will also need to follow the [instructions for setting up the Bonsai.ML.Torch package](../Torch/torch-overview.md) for running on the CPU or GPU.


Rename the package name as above.

glopesdev · 2025-10-24T16:11:02Z

docs/articles/toc.yml

 - name: Overview
-  href: Torch/torch-overview.md
+  href: Torch/torch-overview.md
+- name: Torch.LDS


Same as above to align the naming.

glopesdev · 2025-10-24T16:20:38Z

src/Bonsai.ML.Lds.Torch.Design/ExpectationMaximizationVisualizer.cs

+
+using static TorchSharp.torch;
+
+[assembly: TypeVisualizer(typeof(Bonsai.ML.Lds.Torch.Design.ExpectationMaximizationVisualizer),


Just a thought but given the current decision to lean more on acronyms, this could also be called EMVisualizer.

However, we do have a class already that is named ExpectationMaximization and we probably don't want to contract that to EM, and I do also like the verbose name as it is, so I am torn and more than happy to go either way and leave it as-is.

I suggest we keep the full name for now and we can revisit down the line if we want to change it

glopesdev · 2025-10-24T16:44:17Z

src/Bonsai.ML.Lds.Torch/ILdsState.cs

+/// <summary>
+/// Represents the state of a linear gaussian dynamical system.
+/// </summary>
+public interface ILdsState


I have been wondering whether we should expand Lds inside class names (i.e. leave only the package name with the acronym, but expand inner names).

Just leaving this comment here as the name of this interface was quite opaque to decipher, versus the alternative ILinearDynamicalSystemState.

glopesdev · 2025-10-24T16:47:45Z

src/Bonsai.ML.Lds.Torch/SmoothedState.cs

+/// </summary>
+/// <param name="smoothedMean"></param>
+/// <param name="smoothedCovariance"></param>
+public struct SmoothedState(


Is there a reason we need this specific type? Looks like we might be able to simply use the LdsState class to represent the smoothed state.

glopesdev · 2025-10-24T16:49:01Z

src/Bonsai.ML.Lds.Torch/UpdateParameters.cs

+[ResetCombinator]
+[Description("Updates the parameters of a Kalman filter model instance using the provided Kalman filter parameters.")]
+[WorkflowElementCategory(ElementCategory.Sink)]
+public class UpdateParameters


Suggested change

public class UpdateParameters

public class UpdateKalmanFilterParameters

Should this be renamed for consistency with the Save/Load methods?

…dation during initialization

…ntation

…imating neural latents

…r properties

…ges locally to free up space on test runner

…osed to running both the Python and Bonsai code to save memory

…calSystemState

…avor of using simpler LDS state

…` for improved clarity

…Observations` and `numStates` are not provided

… of a tuple of tensors

…her than individual files which is more consistent with other `Bonsai.ML` packages

…ved validation logic outside of `KalmanFilter` module

… folder instead of individual file paths

…System` struct instead

…Bonsai operator

…g to cpu

…vided by parameters

…including moving devices, setting scalar types, and setting gradient tracking

…property during model creation

…be loaded without explicitly setting type and fixed path support

ncguilbeault added the feature New planned feature label Sep 29, 2025

ncguilbeault force-pushed the dev/torch-lds branch 3 times, most recently from 6b0e89b to 93a8db0 Compare September 30, 2025 16:06

ncguilbeault mentioned this pull request Oct 10, 2025

Examples using the new Bonsai.ML.Lds.Torch package for estimating neural latents bonsai-rx/machinelearning-examples#40

Open

ncguilbeault force-pushed the dev/torch-lds branch from 998bf00 to c4e3bab Compare October 13, 2025 17:23

ncguilbeault force-pushed the dev/torch-lds branch from c4e3bab to 274e6b4 Compare October 13, 2025 17:45

ncguilbeault force-pushed the dev/torch-lds branch 2 times, most recently from fc8b8c0 to 1daacad Compare October 21, 2025 12:28

ncguilbeault mentioned this pull request Oct 23, 2025

GitHub Action Runner Failing Due to Out of Disk Space Exception During Unit Test Execution #75

Open

glopesdev reviewed Oct 24, 2025

View reviewed changes

ncguilbeault force-pushed the dev/torch-lds branch 3 times, most recently from d041405 to f857e16 Compare November 11, 2025 14:29

ncguilbeault force-pushed the dev/torch-lds branch 4 times, most recently from eeb66f6 to a237e2c Compare December 16, 2025 11:24

ncguilbeault added 10 commits December 19, 2025 13:56

Added kalman filter neural latents package to the Bonsai.ML project

3504951

Added Bonsai.ML.Torch project to references

348c51c

Added KalmanFilter class

8c40f3f

Updated KalmanFilter class for better matrix, vector, and scalar vali…

ec8e092

…dation during initialization

Adding classes from original scripted demo

3d369ba

Added test repo for Torch.LDS to compare output with Python impleme…

cf5a2cc

…ntation

Updated Torch.LDS test for running test against Python script for est…

1c43518

…imating neural latents

Added null checks to properties

4905ac7

Updated target framework to include netstandard2.0

55874bf

Updated CreateKalmanFilter class to use TensorConverter for tenso…

6d90172

…r properties

ncguilbeault added 29 commits December 19, 2025 13:56

Updated workflow to correctly update the parameters of the model

1fdd64a

Modified python test script to install packages without caching packa…

85814ee

…ges locally to free up space on test runner

Modified test case to download data and only run Bonsai script as opp…

caabc61

…osed to running both the Python and Bonsai code to save memory

Removed redundant dependency

cdc1a45

Updated documentation with correct package naming

8d1bf46

Changed struct and interface names from LdsState to full LinearDynami…

acf9c6c

…calSystemState

Removed redundant classes for orthogonalized and smoothed states in f…

0f34ab4

…avor of using simpler LDS state

Changed name from UpdateParameters to `UpdateKalmanFilterParameters…

5db9d3c

…` for improved clarity

Updated StateVisualizer to match changes in naming

b59abfe

Fixed test workflow after making changes

a30dc36

Refactored KalmanFilter for improved parameter validation when `num…

6e8645c

…Observations` and `numStates` are not provided

Added overload to create a LinearDynamicalSystemState from a stream…

e9571a3

… of a tuple of tensors

Refactored SaveKalmanFilterParameters class to save to a folder rat…

5280204

…her than individual files which is more consistent with other `Bonsai.ML` packages

Refactored KalmanFilterParameters class to handle validation and mo…

6b3be7e

…ved validation logic outside of `KalmanFilter` module

Updated LoadKalmanFilterParameters operator to load parameters from…

33b762a

… folder instead of individual file paths

Updated test to use null value in NumStates property

4438fe7

Removed unnecessary PredictedState struct and used `LinearDynamical…

b499b76

…System` struct instead

Removed extra check when calling Validate on parameters

f6cf0c0

Fixed issue with setting the incorrect number of iterations from the …

ac99961

…Bonsai operator

Used tensorhelper method to extract float instead of explicitly movin…

b584d55

…g to cpu

Refactored KalmanFilter class to rely on device and scalar type pro…

ce80cf3

…vided by parameters

Refactored KalmanFilterParameters to manage operations on tensors, …

e7add12

…including moving devices, setting scalar types, and setting gradient tracking

Refactored CreateKalmanFilter operator to correctly pass in Type …

2478c08

…property during model creation

Updated to allow specifying Device property when loading parameters

d8deec5

Removed device and scalartype overrides in Initialize method

e44e17d

Refactored LoadKalmanFilterParameters operator to allow tensors to …

77d9b13

…be loaded without explicitly setting type and fixed path support

Updated filtering step to support missing nan values

65674dd

Updated neural latents test to use new load/save tensor method

6260dc1

Updated to use TensorOperatorConverter class

faca4ca

ncguilbeault force-pushed the dev/torch-lds branch from a237e2c to faca4ca Compare December 19, 2025 13:56


		## Installation Guide

		Install the `Bonsai.ML.Torch.LDS` package from the Bonsai package manager. You will also need to follow the [instructions for setting up the Bonsai.ML.Torch package](../Torch/torch-overview.md) for running on the CPU or GPU. No newline at end of file


		using static TorchSharp.torch;

		[assembly: TypeVisualizer(typeof(Bonsai.ML.Lds.Torch.Design.ExpectationMaximizationVisualizer),

	public class UpdateParameters
	public class UpdateKalmanFilterParameters

New package to extract latents from high-dimensional data using LDS and TorchSharp #70

Are you sure you want to change the base?

New package to extract latents from high-dimensional data using LDS and TorchSharp #70

Uh oh!

Conversation

ncguilbeault commented Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

glopesdev commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ncguilbeault commented Oct 13, 2025

Uh oh!

glopesdev left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ncguilbeault commented Sep 29, 2025 •

edited

Loading

glopesdev commented Oct 10, 2025 •

edited

Loading