Skip to content

Conversation

@mattt
Copy link
Owner

@mattt mattt commented Dec 10, 2025

Follow up to #50

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors LlamaLanguageModel to move implementation-specific properties (contextSize, batchSize, threads, seed, temperature, topK, topP, repeatPenalty, repeatLastN) from model-level initialization parameters to per-request custom generation options. This follows up on #50 to improve the API design by allowing these parameters to vary per request rather than being fixed at model initialization.

Key changes:

  • Properties previously set during initialization now default to standard values and can be overridden per request via GenerationOptions custom options
  • Original initializer parameters are deprecated but remain functional (ignoring passed values)
  • New internal ResolvedGenerationOptions struct consolidates option resolution logic
  • Refactored sampling logic into a dedicated applySampling method

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
Sources/AnyLanguageModel/Models/LlamaLanguageModel.swift Adds custom option properties to CustomGenerationOptions, deprecates model-level properties, introduces ResolvedGenerationOptions for option resolution, refactors initializers and generation methods to use resolved options, extracts sampling logic into applySampling method
Tests/AnyLanguageModelTests/LlamaLanguageModelTests.swift Updates tests to reflect new initialization API, adds tests for custom options round-trip and deprecated initializer behavior, expands test for custom generation options with new parameters

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mattt mattt merged commit 1fa6629 into main Dec 10, 2025
3 checks passed
@mattt mattt deleted the mattt/llama-generation-options branch December 10, 2025 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants