-
Notifications
You must be signed in to change notification settings - Fork 42
Update LlamaLanguageModel to replace implementation-specific properties with custom generation options #56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…es with custom generation options
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR refactors LlamaLanguageModel to move implementation-specific properties (contextSize, batchSize, threads, seed, temperature, topK, topP, repeatPenalty, repeatLastN) from model-level initialization parameters to per-request custom generation options. This follows up on #50 to improve the API design by allowing these parameters to vary per request rather than being fixed at model initialization.
Key changes:
- Properties previously set during initialization now default to standard values and can be overridden per request via
GenerationOptionscustom options - Original initializer parameters are deprecated but remain functional (ignoring passed values)
- New internal
ResolvedGenerationOptionsstruct consolidates option resolution logic - Refactored sampling logic into a dedicated
applySamplingmethod
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| Sources/AnyLanguageModel/Models/LlamaLanguageModel.swift | Adds custom option properties to CustomGenerationOptions, deprecates model-level properties, introduces ResolvedGenerationOptions for option resolution, refactors initializers and generation methods to use resolved options, extracts sampling logic into applySampling method |
| Tests/AnyLanguageModelTests/LlamaLanguageModelTests.swift | Updates tests to reflect new initialization API, adds tests for custom options round-trip and deprecated initializer behavior, expands test for custom generation options with new parameters |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Follow up to #50