Skip to content

Conversation

@neo-jesse
Copy link
Collaborator

@neo-jesse neo-jesse commented Nov 12, 2025

Migrate Python dependency management from Pipenv to UV

Summary

Migrates api and shared-data projects from pipenv to uv for faster dependency resolution and installation. All other projects continue using pipenv.

Key Changes for Api and shared-data

  • Migrated api and shared-data from Pipfile to pyproject.toml with uv.lock
  • Created scripts/python-uv.mk for UV projects; scripts/python.mk remains for pipenv projects
  • Updated GitHub Actions setup action to detect project type (UV vs pipenv) and install dependencies accordingly
  • Fixed dependencies: added opentrons-hardware[FLEX] to api dev deps, numpy to shared-data main deps
  • Fixed mypy errors (redundant casts, unused ignores, untyped decorators)
  • Migrated JSON schema validation from jsonschema.RefResolver to referencing library
  • Refactored GitHub Actions Python setup into modular actions (install-uv, cache-uv, setup)
  • Fixed Windows PATH issues for uv in Make commands
  • Added install-dev-deps: 'true' to workflows that run lint/test for UV projects

Changes for Pipenv Projects

  • Updated all pipenv Makefiles to use pipenv sync --dev to ensure dev dependencies are installed
  • Fixed workflows to properly install dev dependencies for lint/test jobs

Benefits

  • Faster dependency resolution and installation for api and shared-data
  • Standardized pyproject.toml configuration for migrated projects
  • Improved CI reliability with proper caching and dev dependency handling
  • Gradual migration path - other projects can be migrated incrementally

@codecov
Copy link

codecov bot commented Nov 12, 2025

Codecov Report

❌ Patch coverage is 0.33727% with 591 lines in your changes missing coverage. Please review.
✅ Project coverage is 5.59%. Comparing base (bd8f6eb) to head (f401f0d).
⚠️ Report is 106 commits behind head on edge.

Files with missing lines Patch % Lines
.../Desktop/ProtocolVisualization/ModuleContainer.tsx 0.00% 164 Missing ⚠️
app/src/local-resources/runs/utils.ts 0.00% 76 Missing ⚠️
app/src/local-resources/runs/constants.ts 0.00% 75 Missing ⚠️
...c/organisms/Desktop/Devices/RecentProtocolRuns.tsx 0.00% 75 Missing ⚠️
.../organisms/Desktop/Camera/CameraControls/index.tsx 0.00% 35 Missing ⚠️
app-shell/scripts/python-install.js 0.00% 19 Missing ⚠️
...-resources/images/hooks/useCameraSettingsValues.ts 0.00% 17 Missing ⚠️
app/src/molecules/SlotDetailsEmptyState/index.tsx 0.00% 13 Missing ⚠️
...ProtocolRun/SetupCamera/SetupRunCameraControls.tsx 0.00% 10 Missing ⚠️
...rganisms/Desktop/Devices/InstrumentsAndModules.tsx 0.00% 9 Missing ⚠️
... and 25 more
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##             edge   #20135       +/-   ##
===========================================
- Coverage   24.80%    5.59%   -19.22%     
===========================================
  Files        3621     3262      -359     
  Lines      304334   254627    -49707     
  Branches    43469     5414    -38055     
===========================================
- Hits        75495    14244    -61251     
- Misses     228816   240383    +11567     
+ Partials       23        0       -23     
Flag Coverage Δ
app ?
components ?
hardware ?
labware-library ?
opentrons-ai-client ?
protocol-designer ?
react-api-client ?
shared-data ?
step-generation 5.59% <0.33%> (+0.09%) ⬆️
system-server ?
update-server ?
usb-bridge ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...op/Devices/ProtocolRun/ProtocolRunCamera/index.tsx 0.00% <ø> (ø)
...lRunHeader/RunHeaderContent/ActionButton/index.tsx 0.00% <ø> (ø)
...nHeader/RunHeaderContent/RunHeaderSectionUpper.tsx 0.00% <ø> (ø)
...rotocolRun/ProtocolRunHeader/hooks/useRunErrors.ts 0.00% <ø> (ø)
...top/Devices/ProtocolRun/ProtocolRunHeader/utils.ts 0.00% <ø> (ø)
.../RobotSettings/AdvancedTab/UpdateRobotSoftware.tsx 0.00% <ø> (ø)
app-shell/src/secondary-windows/index.ts 0.00% <0.00%> (ø)
...HeaderModalContainer/hooks/useMissingStepsModal.ts 0.00% <0.00%> (ø)
...nHeaderModalContainer/hooks/useRunHeaderDropTip.ts 0.00% <0.00%> (ø)
...HeaderModalContainer/modals/ConfirmCancelModal.tsx 0.00% <0.00%> (ø)
... and 31 more

... and 1994 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@neo-jesse neo-jesse marked this pull request as ready for review November 13, 2025 17:40
@neo-jesse neo-jesse requested review from a team as code owners November 13, 2025 17:40
@neo-jesse neo-jesse requested review from ncdiehl11 and removed request for a team November 13, 2025 17:40
Copy link
Member

@sfoster1 sfoster1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's try and do all we can to avoid changes that aren't necessary for the uv cutover. that's stuff like

  • Altering the pipenv_opts definition requiring changes to all the makefiles to push --dev up
  • Whatever is causing the typechecking to change and requiring some extra casts
  • Whatever made us reformat ot3controller.py

There's also some other little stuff:

  • is there ever a time we would not want to install dev dependencies? can we just remove that argument to the action, or invert it to no-install-dev-deps and specify it more rarely?
  • let's rely on python depspecs for the linux only dependencies
  • some code duplication around path specification i think we can pull
  • let's just double check that the versions of all the dependencies, including dev dependencies, are the same in the new uv and the old pipenv dependency specs. that will let us change fewer things at one time

id: project-type
shell: bash
run: |
# Only api and shared-data use UV for now
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we probe for the existence of a uv.lock file instead, or even cut over the makefile

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More than just the 2 projects we are looking at have uv.lock, it's not a good indicator.

Copy link
Member

@sfoster1 sfoster1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very much getting there! A couple little nits to pick; the only big things left are some github actions stuff, the thing with the mypy configuration in api, and a weirdness with the dependencies in api.

@sfoster1 sfoster1 changed the title feat(uv, ci): Api and shared-data uv migration chore: python project migration to uv Dec 9, 2025
Added imports for referencing

added --dev back

referencing
more squashed commits
@neo-jesse neo-jesse force-pushed the api-and-shared-uv-migration branch from ab14763 to 433422a Compare December 9, 2025 20:25
koji and others added 4 commits December 9, 2025 13:05
* fix(app): fix command step height issue
…20208)

* fix(components): fix timeline scrubber's background hover behavior
…art of QuickTransferSummaryState (#20212)

# Overview

This is a type-hinting change to tell TypeScript that the
`aspirateState`/`dispenseState` in `retrieveLiquidClassValues.ts` is
part of `QuickTransferSummaryState`.

Without this, VS Code doesn't find this file when we're looking for
places that touch `.disposalVolumeDispenseSettings`, wasting a lot of
time for programmers trying to find what is setting that field.

Adding this annotation also helps TypeScript catch typos. For example,
if this file had accidentally misspelled the field as
`disposalVolumeDispenseSettingsxx`, TypeScript would now complain about
it.

## Test Plan and Hands on Testing

Just run CI.

## Risk assessment

Very low. No functional change whatsoever.
Correctly use `set_load_module` for StateUpdate when mapping 2.13 and below PAPI load module calls
sfoster1 and others added 29 commits December 9, 2025 13:25
The other projecs use more recent flake8, let's do it here too.
)

This PR was requested on the PR
#20109

Co-authored-by: sfoster1 <3091648+sfoster1@users.noreply.github.com>
With the decision to treat Quick Transfer protocols more like standard
protocols, we want to show them in the desktop app's
`RecentProtocolRuns` component. This also allows us to view and download
any files from the run, which as of now would include photos taken
during any potential error recovery.

There are also a few instances in ODD where we implicitly delete QT
runs. We don't want to do this!
# Overview

Moved run status utils into `local-resources` since they are used across
platforms in the app
Replaced the majority of instances of bare `runStatus` strings with
utils

## Changelog
- moved utils from `ProtocolHeader` to `local-resources`
- fixed conditional bug that was preventing you from restarting a run
due to the robot being in idle state

## Test Plan and Hands on Testing
- smoke tested various places of app and odd

## Review requests

- Double check run status groupings aren't repeated
- Double check majority of run status bare string usage is addressed


## Risk assessment

medium - touches a lot of code but does not change functionality

Closes second half of EXEC-1478
…ls (#20197)

# Overview

Camera, concurrent modules, and dynamic liquid tracking implemented into
active ABR protocols

## Test Plan and Hands on Testing

- All protocols simulate
- `TODO:` protocols with the tc lid and dynamic liquid tracking have not
been tested on a robot yet

## Changelog
- also added a robot log script to compare duration of protocol commands

## Review requests

@ryanthecoder when implementing the dynamic pipetting, there were a
couple spots I needed to add `prepare_to_aspirate()` before a dynamic
aspirate to avoid getting this error:

```
ProtocolCommandFailedError [line 1035]: Error 4000 GENERAL_ERROR (ProtocolCommandFailedError): PipetteNotReadyToAspirateError: Pipette cannot aspirate while tracking because a previous dispense or blowout pushed the plunger beyond the bottom position. The subsequent aspirate must be from a specific well so the plunger can be reset in a known safe position.
```
Is this expected?

## Risk assessment

medium until the protocols are run on the robot
…ges (#20299)

Closes RQA-4941

Axios cannot return Blob types when in a Node.js environment (there is no native blob type in the version of Axios we use), and Electron's IPC mechanism does not properly serialize Blob objects into strings anyway.

To fix, on the node layer we can request an ArrayBuffer type instead, transform it to an Array type, and then send this data across the IPC as it becomes perfectly serializable. We can then parse this Array type back into a Blob on the browser layer.

Note that downloading images works perfectly fine - it's just an issue whenever we request the raw image files themselves.
* feat(app): add Skeleton component for second window
* fix(app): remove number from individual command
# Overview

Add non functional livestream image capture button hidden behind the
camera settings feature flag

## Test Plan and Hands on Testing

Smoke tested on ODD w/o feature flag on
<img width="491" height="430" alt="Screenshot 2025-12-04 at 2 56 59 PM"
src="https://github.com/user-attachments/assets/750a5315-4eee-4dbc-8a8d-698d8784683a"
/>

## Review requests



## Risk assessment

low

Closes EXEC-2101
…are (#20253)

# Overview

PR for touch probe proof of concept. Adds firmware, drivers, and
hardware testing scripts for the touch probe.


https://docs.google.com/document/d/151cAolVlUmFbdYdXOCe5GIgDmzycKI_iPKSS7zWbS0I/edit?tab=t.0

## Changelog

-added touch probe firmware + drivers
-added touch_probe mock scripts to hardware testing

---------

Co-authored-by: Ryan Howard <ryan.howard@opentrons.com>
Co-authored-by: Rhyann Clarke <146747548+rclarke0@users.noreply.github.com>
Co-authored-by: Rhyann Clarke <rhyann.clarke@opentrons.com>
#20311)

* fix(components): fix primarybutton and alt primarybutton styling issue
…are (#20253)

PR for touch probe proof of concept. Adds firmware, drivers, and
hardware testing scripts for the touch probe.

https://docs.google.com/document/d/151cAolVlUmFbdYdXOCe5GIgDmzycKI_iPKSS7zWbS0I/edit?tab=t.0

-added touch probe firmware + drivers
-added touch_probe mock scripts to hardware testing

---------

Co-authored-by: Ryan Howard <ryan.howard@opentrons.com>
Co-authored-by: Rhyann Clarke <146747548+rclarke0@users.noreply.github.com>
Co-authored-by: Rhyann Clarke <rhyann.clarke@opentrons.com>
Numpy 1.26.4 doesn't have a precompiled wheel for python 3.13, so when
you install it pip will build it locally. This takes a while, but also
creates a bad object on windows since numpy doesn't support mingw.
more squashed commits
@neo-jesse neo-jesse requested a review from a team as a code owner December 9, 2025 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.