Skip to content

Conversation

@mattt
Copy link
Owner

@mattt mattt commented Dec 10, 2025

No description provided.

@mattt mattt force-pushed the mattt/llama-chat-template branch from 39a76c3 to acafecf Compare December 10, 2025 17:17
@mattt mattt requested a review from Copilot December 10, 2025 17:17
@mattt mattt marked this pull request as ready for review December 10, 2025 17:17
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements chat template support for LlamaLanguageModel, enabling proper multi-turn conversation formatting using the model's embedded chat templates (e.g., Llama 3, Mistral, ChatML). Instead of using raw prompt text, the model now applies conversation-aware formatting by extracting the full transcript history and applying the GGUF-embedded chat template via llama.cpp's llama_chat_apply_template API.

Key Changes:

  • Adds formatPrompt(for:) method to convert transcript entries into formatted prompts using model-embedded chat templates
  • Adds extractText(from:) helper to extract text content from transcript segments
  • Updates both respond and streamResponse methods to use formatted prompts instead of raw prompt descriptions

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mattt mattt merged commit 14a29c1 into main Dec 10, 2025
9 checks passed
@mattt mattt deleted the mattt/llama-chat-template branch December 10, 2025 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants