Skip to content

Conversation

@mathiusj
Copy link
Contributor

@mathiusj mathiusj commented Dec 9, 2025

TL;DR

Part of: #570

Added VCR support for DSL workflow tests to enable reliable HTTP request recording and playback.

What changed?

  • Enhanced the in_sandbox method in FunctionalTest to wrap execution in VCR cassettes
  • Added configuration for RubyLLM to work with VCR during tests
  • Created a new test case "VCR infrastructure works for DSL chat workflows" to verify functionality
  • Added a VCR cassette file with a mock response about Lake Baikal
  • Created a simple test workflow file vcr_test.rb that makes an LLM API call

How to test?

  1. Run the new test: OPENAI_API_KEY=api-key-here RECORD_VCR=true bundle exec ruby -Itest test/dsl/functional/roast_dsl_examples_test.rb
  2. Verify that tests pass without real API calls when using the recorded cassettes

Why make this change?

This change improves test reliability by eliminating dependencies on external API services during testing. By recording API interactions once and playing them back during subsequent test runs, we can:

  • Make tests faster and more deterministic
  • Avoid hitting rate limits or incurring costs from real API calls
  • Test without requiring real API credentials
  • Ensure consistent test behavior regardless of network conditions

Copy link
Contributor Author

mathiusj commented Dec 9, 2025

This stack of pull requests is managed by Graphite. Learn more about stacking.

@mathiusj mathiusj force-pushed the mathiusj/gh-570-add-vcr-support-dsl-tests branch from 8a0efda to 0125231 Compare December 9, 2025 22:22
Copy link
Contributor Author

mathiusj commented Dec 9, 2025

@juniper-shopify updated approach to add VCR support for the functional tests. I can iterate on this a bit more if this looks right to you 🙏

@mathiusj mathiusj force-pushed the mathiusj/gh-570-add-vcr-support-dsl-tests branch from 0125231 to 61ae048 Compare December 9, 2025 23:59
@mathiusj mathiusj force-pushed the mathiusj/gh-570-add-vcr-support-dsl-tests branch from 61ae048 to 30df328 Compare December 10, 2025 00:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant