TTS/STT: Speech-To-Text Using Gemini in Unified API #550
+3,401
−735
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
📝 WalkthroughWalkthroughThis PR introduces Google Gemini STT provider support, partial configuration version updates, and comprehensive LLM call tracking infrastructure. New GoogleAIProvider implementation with input resolution for multimodal inputs, ConfigVersionCreatePartial for incremental config updates with type-preservation validation, LlmCall model with full CRUD operations for tracking LLM invocations, and updated provider registry to support multiple providers uniformly. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant JobsService
participant InputResolver
participant LLMProvider
participant LLMCrud
participant Database
Client->>JobsService: execute_job(job, config)
JobsService->>InputResolver: resolve_input(query_input)
InputResolver-->>JobsService: resolved_input (text or file path)
JobsService->>LLMCrud: create_llm_call(request, config)
LLMCrud->>Database: INSERT llm_call
Database-->>LLMCrud: llm_call_id
LLMCrud-->>JobsService: LlmCall record
JobsService->>LLMProvider: execute(config, query, resolved_input)
LLMProvider-->>JobsService: LLMCallResponse
JobsService->>LLMCrud: update_llm_call_response(llm_call_id, content, usage)
LLMCrud->>Database: UPDATE llm_call
Database-->>LLMCrud: updated LlmCall
LLMCrud-->>JobsService: LlmCall with response
JobsService->>InputResolver: cleanup_temp_file(file_path)
InputResolver-->>JobsService: cleaned up
JobsService-->>Client: job result
sequenceDiagram
participant Client
participant ConfigRoute
participant ConfigVersionCrud
participant Database
Client->>ConfigRoute: PATCH /config/{id}/version (partial blob)
ConfigRoute->>ConfigVersionCrud: create_from_partial_or_raise(partial_data)
ConfigVersionCrud->>Database: SELECT latest version
Database-->>ConfigVersionCrud: latest ConfigVersion
ConfigVersionCrud->>ConfigVersionCrud: _deep_merge(latest.blob, partial)
ConfigVersionCrud->>ConfigVersionCrud: _validate_immutable_fields(latest, merged)
alt Type mismatch
ConfigVersionCrud-->>ConfigRoute: ValidationError
ConfigRoute-->>Client: 400 Bad Request
else Type preserved
ConfigVersionCrud->>Database: INSERT new version
Database-->>ConfigVersionCrud: new ConfigVersion
ConfigVersionCrud-->>ConfigRoute: ConfigVersion
ConfigRoute-->>Client: 200 OK (updated config)
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related issues
Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…-stt-new' into feature/unified-api-stt-new merge upstream
Prajna1999
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 19
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (5)
backend/app/celery/beat.py (1)
12-21: Add return type and log prefix instart_beat.
Missing return annotation and required log prefix.✅ Suggested fix
-def start_beat(loglevel: str = "info"): +def start_beat(loglevel: str = "info") -> None: @@ - logger.info(f"Starting Celery beat scheduler") + logger.info("[start_beat] Starting Celery beat scheduler")backend/app/tests/services/doctransformer/test_job/utils.py (1)
8-99: Add missing type hints for helper factories and callables.
Strict typing will flag these functions (and inner callables) without explicit annotations.✅ Suggested fix
-from pathlib import Path +from pathlib import Path +from typing import Any, Callable, NoReturn from urllib.parse import urlparse @@ -def create_failing_convert_document(fail_count: int = 1): +def create_failing_convert_document(fail_count: int = 1) -> Callable[..., Path]: @@ - def failing_convert_document(*args, **kwargs): + def failing_convert_document(*args: Any, **kwargs: Any) -> Path: @@ -def create_persistent_failing_convert_document( - error_message: str = "Persistent error", -): +def create_persistent_failing_convert_document( + error_message: str = "Persistent error", +) -> Callable[..., NoReturn]: @@ - def persistent_failing_convert_document(*args, **kwargs): + def persistent_failing_convert_document(*args: Any, **kwargs: Any) -> NoReturn: raise Exception(error_message)backend/app/celery/utils.py (1)
19-111: Add**kwargstype hints and prefix log messages with function names.
This aligns with strict typing and logging guidelines.✅ Suggested fix
def start_high_priority_job( - function_path: str, project_id: int, job_id: str, trace_id: str = "N/A", **kwargs + function_path: str, + project_id: int, + job_id: str, + trace_id: str = "N/A", + **kwargs: Any, ) -> str: @@ - logger.info(f"Started high priority job {job_id} with Celery task {task.id}") + logger.info( + f"[start_high_priority_job] Started high priority job {job_id} with Celery task {task.id}" + ) @@ def start_low_priority_job( - function_path: str, project_id: int, job_id: str, trace_id: str = "N/A", **kwargs + function_path: str, + project_id: int, + job_id: str, + trace_id: str = "N/A", + **kwargs: Any, ) -> str: @@ - logger.info(f"Started low priority job {job_id} with Celery task {task.id}") + logger.info( + f"[start_low_priority_job] Started low priority job {job_id} with Celery task {task.id}" + ) @@ - logger.info(f"Revoked task {task_id}") + logger.info(f"[revoke_task] Revoked task {task_id}") return True except Exception as e: - logger.error(f"Failed to revoke task {task_id}: {e}") + logger.error(f"[revoke_task] Failed to revoke task {task_id}: {e}") return Falsebackend/app/celery/worker.py (1)
14-41: Fix typing forconcurrency/return and add log prefixes.
Ensures strict typing and consistent log formatting.✅ Suggested fix
def start_worker( queues: str = "default,high_priority,low_priority,cron", - concurrency: int = None, + concurrency: int | None = None, loglevel: str = "info", -): +) -> None: @@ - logger.info(f"Starting Celery worker with {concurrency} processes") - logger.info(f"Consuming queues: {queues}") + logger.info(f"[start_worker] Starting Celery worker with {concurrency} processes") + logger.info(f"[start_worker] Consuming queues: {queues}")backend/app/services/llm/mappers.py (1)
7-76: Handle potentialNonemodel inlitellm.supports_reasoningcall.On Line 39, if
modelisNone(fromkaapi_params.get("model")), the calllitellm.supports_reasoning(model=f"openai/{model}")will pass"openai/None"which may cause unexpected behavior or errors from litellm.🐛 Proposed fix
model = kaapi_params.get("model") reasoning = kaapi_params.get("reasoning") temperature = kaapi_params.get("temperature") instructions = kaapi_params.get("instructions") knowledge_base_ids = kaapi_params.get("knowledge_base_ids") max_num_results = kaapi_params.get("max_num_results") - support_reasoning = litellm.supports_reasoning(model=f"openai/{model}") + support_reasoning = ( + litellm.supports_reasoning(model=f"openai/{model}") if model else False + )
🤖 Fix all issues with AI agents
In `@backend/app/models/llm/request.py`:
- Around line 4-11: Remove the duplicate import of Field and SQLModel:
consolidate the two sqlmodel import lines into one (e.g., replace the two
occurrences of "from sqlmodel import Field, SQLModel" with a single line that
also includes Index and text if needed: "from sqlmodel import Field, SQLModel,
Index, text"), leaving other imports (pydantic, datetime, sqlalchemy, JSONB,
app.core.util) unchanged; ensure only one import statement provides Field and
SQLModel to fix the Ruff F811 duplicate-import error.
- Around line 313-479: The updated_at field on the LlmCall model currently uses
default_factory=now so it only sets at creation; make it auto-update on
modifications by adding an SQLAlchemy onupdate to the Column (e.g., sa_column or
sa_column_kwargs for updated_at with onupdate=now) or, if you prefer
application-level handling, ensure the update_llm_call_response CRUD function
(or any updater) sets updated_at = now() before committing. Update the
LlmCall.updated_at definition accordingly and/or modify update_llm_call_response
to assign now() on each update so updated_at reflects the last modification.
In `@backend/app/services/llm/input_resolver.py`:
- Around line 86-111: The resolve_audio_url function currently fetches arbitrary
URLs; before calling requests.get in resolve_audio_url, validate the input URL
by reusing validate_callback_url(url) (or at minimum enforce scheme == "https"
and use _is_private_ip to reject private/link-local IPs) and return an error
string on validation failure; also call requests.get with allow_redirects=False
to disable redirects and keep the existing timeout; keep existing temp file
write logic (references: resolve_audio_url, validate_callback_url,
_is_private_ip, get_file_extension).
In `@backend/app/services/llm/jobs.py`:
- Line 212: Replace the print call printing completion_config with a logger call
using the module logger (e.g., logger.debug or logger.info) instead of print;
log the same message text but prefixed with the current function name and
include the completion_config variable for context (in
backend/app/services/llm/jobs.py, at the location where completion_config is
printed), ensuring you use the existing module logger (or create one via
logging.getLogger(__name__)) and the appropriate log level rather than print.
- Around line 160-178: Remove the temporary debug block in execute_job that
performs an inline import of select and queries recent jobs when
session.get(Job, job_id) returns None; delete the extra session.exec(...) query,
the inline "from sqlmodel import select" import, and the verbose logger.error
that prints recent jobs, leaving only the initial logger.info that attempts to
fetch the job and the existing logger.error (or add a concise logger.error) for
the missing Job; ensure any needed diagnostics are moved to a dedicated utility
function (e.g., a new diagnostic helper) rather than inline in execute_job.
- Around line 299-302: The cleanup currently compares resolved_input (str) to
request.query.input (QueryInput) which is always true; change the finally block
to only call cleanup_temp_file(resolved_input) when the original input is an
audio-type input that created a temp file — e.g., check
isinstance(request.query.input, (AudioBase64Input, AudioUrlInput)) and
resolved_input is truthy before calling cleanup_temp_file; leave TextInput alone
so we don't attempt to treat text as a temp file.
In `@backend/app/services/llm/PLAN.md`:
- Around line 222-225: The example for the Field definition for conversation_id
has a missing comma after default=None causing a syntax error; update the
conversation_id Field invocation (the conversation_id variable and its use of
Field) to include the missing comma between default=None and
description="Identifier linking this response to its conversation thread" so the
Field call is properly separated and the snippet parses correctly.
- Around line 113-115: The log message in the example uses the wrong provider
name: update the logger.info call that currently says "[OpenAIProvider.execute]
Successfully generated response: {response.response_id}" to reference the
correct provider and method (e.g., "[GoogleAIProvider.execute] Successfully
generated response: {response.response_id}") so the log reflects
GoogleAIProvider.execute; locate the logger.info in the GoogleAIProvider.execute
example and change only the provider name in the message.
In `@backend/app/services/llm/providers/gai.py`:
- Around line 75-78: The lang_instruction assignment in the block that checks
input_language uses an unnecessary f-string prefix; update the two assignments
so they are plain strings (remove the leading 'f') for the branches where you
set lang_instruction (the conditional using input_language and the variable
lang_instruction).
- Around line 38-43: The _parse_input implementation only handles
completion_type "stt" and lacks type hints, causing implicit None returns;
update the method signature to include type hints (e.g., def _parse_input(self,
query_input: Any, completion_type: str, provider: str) -> str) and implement
explicit handling for non-"stt" completion types: validate and coerce/return a
string for other expected types (e.g., "chat" or "text"/"completion") or raise a
clear ValueError when the input shape is invalid; ensure every control path
returns a str and import any typing symbols used.
- Around line 32-36: OpenAIProvider.create_client currently returns an error
string when credentials are missing, causing a wrong type to be passed to the
constructor; change it to raise an exception instead to match
GoogleAIProvider.create_client's behavior (which raises ValueError). Update
OpenAIProvider.create_client to check for the required credential key (e.g.,
"api_key" or provider-specific name) and raise a ValueError with a clear message
when missing so the registry's exception handler receives an exception rather
than a string.
- Around line 55-125: The execute method in GoogleAIProvider only handles
completion_type == "stt" and falls through silently for other types; update
execute to explicitly handle unsupported completion types (e.g., "text" and
"tts") by returning a clear error (or implementing their flows) when
completion_type is not "stt". Locate the block using completion_type,
completion_config, and the STT flow where gemini_file/upload and
client.models.generate_content are used, and add an else branch (or early guard)
that returns (None, "<descriptive error>") or raises a descriptive exception
indicating unsupported completion_type so callers no longer get an implicit
(None, None).
In `@backend/app/services/llm/providers/oai.py`:
- Around line 32-36: The create_client staticmethod in OpenAIProvider
(create_client) currently returns an error string and uses an unnecessary
f-string; change it to mirror GoogleAIProvider.create_client by raising a
ValueError when "api_key" is missing (so callers receive an exception instead of
a string), and replace the f-string with a plain string literal; ensure the
method otherwise returns the OpenAI(...) client instance to keep the return type
consistent.
In `@backend/app/services/llm/providers/registry.py`:
- Around line 92-135: Remove the ad-hoc "__main__" test block from registry.py;
the block contains hardcoded paths and an outdated call signature. Extract the
logic that uses LLMProvider.get_provider_class, ProviderClass.create_client,
NativeCompletionConfig, QueryParams and instance.execute into a proper test
under backend/app/tests/services/llm/providers/test_gai.py (or delete it),
update the execute invocation to include the resolved_input parameter to match
the current signature, and ensure any credential/env handling is mocked rather
than reading real env vars or local file paths.
- Around line 66-70: There is a duplicated assignment of credential_provider
from provider_type using replace("-native", ""); remove the redundant line so
credential_provider is assigned only once (keep the first or the clearer
occurrence) and ensure any surrounding comments remain correct; locate the
duplicate by searching for the variable name credential_provider and the
expression provider_type.replace("-native", "") in registry.py and delete the
extra assignment.
- Around line 13-26: Remove the testing artifacts imported into the module:
delete the temporary import "from google.genai.types import
GenerateContentConfig", the block importing NativeCompletionConfig,
LLMCallResponse, QueryParams, LLMOutput, LLMResponse, Usage from app.models.llm
(if they are unused here), and the call to load_dotenv(); ensure any genuinely
required symbols for functions/classes in this file (e.g., registry-related
classes or functions) remain imported from their proper modules and move
environment loading to application startup code rather than leaving
load_dotenv() in this module.
In `@backend/app/tests/crud/test_llm.py`:
- Around line 269-271: The test passes an integer literal 99999 as project_id to
get_llm_call_by_id but project_id is a UUID; change the test to pass a UUID that
will not match the created LLM call (e.g., generate a new UUID via uuid.uuid4()
or use a different fixture UUID) instead of 99999 so the call uses the correct
type; update the assertion code around get_llm_call_by_id(db, created.id,
project_id=...) and ensure imports/fixtures provide a UUID value.
In `@backend/app/tests/services/llm/test_mappers.py`:
- Around line 1-317: The entire test suite in
backend/app/tests/services/llm/test_mappers.py is commented out; restore test
visibility by either re-enabling the tests or explicitly skipping them with a
reason and tracking link. Undo the block comment (or reintroduce the classes
TestMapKaapiToOpenAIParams and TestTransformKaapiConfigToNative) and run/fix
failing assertions against map_kaapi_to_openai_params and
transform_kaapi_config_to_native if they break due to the new type system, or if
you must temporarily disable, add pytest.mark.skip(reason="TODO: update tests
for new type system, see ISSUE-XXXXX") above each Test* class and add a TODO
comment referencing the issue; ensure the skip preserves the original test names
and imports so future fixes can target the exact failing assertions.
In `@backend/app/tests/utils/test_data.py`:
- Around line 339-373: latest_version may lack a "type" in its config_blob so
config_type can be None and later fail Literal validation; set a sensible
default (e.g., "text") when extracting it from completion_config and use that
default when constructing ConfigBlob instances for both NativeCompletionConfig
and KaapiCompletionConfig (update the assignment of config_type taken from
completion_config.get("type") to fallback to "text" and ensure
NativeCompletionConfig and KaapiCompletionConfig are always passed a non-None
type).
🧹 Nitpick comments (27)
backend/app/tests/scripts/test_backend_pre_start.py (1)
5-24: Add return type hint to the test function.Per coding guidelines, all functions should have type hints for parameters and return values.
Suggested fix
-def test_init_success(): +def test_init_success() -> None:backend/app/tests/scripts/test_test_pre_start.py (1)
5-24: Add return type hint to the test function.Per coding guidelines, all functions should have type hints for parameters and return values.
Suggested fix
-def test_init_success(): +def test_init_success() -> None:backend/app/cli/bench/commands.py (1)
169-175: UseCallablefromtypinginstead of lowercasecallable.The
callableon line 174 is a built-in function, not a type annotation. For proper static type checking, useCallablefrom thetypingmodule with the appropriate signature.Suggested fix
-from typing import List, Protocol +from typing import Callable, List, Protocoldef send_benchmark_request( prompt: str, i: int, total: int, endpoint: str, - build_payload: callable, + build_payload: Callable[[str], dict], ) -> BenchItem:As per coding guidelines, type hints should be used throughout the codebase.
backend/app/tests/services/llm/providers/test_registry.py (1)
23-26: Consider adding test coverage for GoogleAIProvider in registry.The registry tests only verify
openai-nativeprovider. With the addition ofGoogleAIProvider, consider adding a test to verifygoogle-nativeis also registered correctly inLLMProvider._registry.backend/app/models/config/version.py (1)
99-116: Consider adding validation for empty config_blob.
ConfigVersionBaseincludes avalidate_blob_not_emptyvalidator (line 32-36), butConfigVersionCreatePartialdoesn't inherit from it or define its own validator. If passing an emptyconfig_blobin a partial update is invalid, consider adding validation:from pydantic import field_validator `@field_validator`("config_blob") def validate_blob_not_empty(cls, value): if not value: raise ValueError("config_blob cannot be empty") return valueIf empty is intentionally allowed (e.g., the CRUD layer handles merging with existing config), this can be ignored.
backend/app/services/llm/__init__.py (1)
1-6: Consider consolidating imports from the same module.Both import statements pull from
app.services.llm.providers. They could be combined into a single import for cleaner organization:♻️ Suggested refactor
-from app.services.llm.providers import BaseProvider, OpenAIProvider, GoogleAIProvider -from app.services.llm.providers import ( - LLMProvider, - get_llm_provider, -) +from app.services.llm.providers import ( + BaseProvider, + GoogleAIProvider, + LLMProvider, + OpenAIProvider, + get_llm_provider, +)backend/app/tests/api/routes/test_llm.py (1)
145-167: Consider adding thetypefield to the invalid provider test payload.The
test_llm_call_invalid_providertest payload is missing thetypefield in the completion config. While this may still trigger a 422 due to the invalid provider, it could mask validation order issues. Consider adding"type": "text"to ensure the test specifically validates provider validation.Suggested fix
payload = { "query": {"input": "Test query"}, "config": { "blob": { "completion": { "provider": "invalid-provider", + "type": "text", "params": {"model": "gpt-4"}, } } }, }backend/app/api/routes/config/version.py (4)
25-45: Add return type hint tocreate_versionfunction.Per coding guidelines, all function parameters and return values should have type hints. The function returns
APIResponse[ConfigVersionPublic].Suggested fix
def create_version( config_id: UUID, version_create: ConfigVersionCreatePartial, current_user: AuthContextDep, session: SessionDep, -): +) -> APIResponse[ConfigVersionPublic]:
55-75: Add return type hint tolist_versionsfunction.Per coding guidelines, all functions should have return type hints.
Suggested fix
def list_versions( config_id: UUID, current_user: AuthContextDep, session: SessionDep, skip: int = Query(0, ge=0, description="Number of records to skip"), limit: int = Query(100, ge=1, le=100, description="Maximum records to return"), -): +) -> APIResponse[list[ConfigVersionItems]]:
85-102: Add return type hint toget_versionfunction.Per coding guidelines, all functions should have return type hints.
Suggested fix
def get_version( config_id: UUID, current_user: AuthContextDep, session: SessionDep, version_number: int = Path( ..., ge=1, description="The version number of the config" ), -): +) -> APIResponse[ConfigVersionPublic]:
112-130: Add return type hint todelete_versionfunction.Per coding guidelines, all functions should have return type hints.
Suggested fix
def delete_version( config_id: UUID, current_user: AuthContextDep, session: SessionDep, version_number: int = Path( ..., ge=1, description="The version number of the config" ), -): +) -> APIResponse[Message]:backend/app/tests/crud/test_llm.py (2)
26-39: Consider using factory pattern for test fixtures.Per coding guidelines, test fixtures in
backend/app/tests/should use the factory pattern. These fixtures query seed data directly rather than using factories. Consider creating factory functions for test projects and organizations if they don't already exist.
42-46: Add return type hint totest_jobfixture.Per coding guidelines, all functions should have type hints.
Suggested fix
`@pytest.fixture` -def test_job(db: Session): +def test_job(db: Session) -> Job: """Create a test job for LLM call tests.""" crud = JobCrud(db) return crud.create(job_type=JobType.LLM_API, trace_id="test-llm-trace")Note: You'll need to import
Jobfrom the appropriate models module.backend/app/tests/services/llm/test_jobs.py (1)
19-21: Remove commented-out import instead of leaving it.The
KaapiLLMParamsimport is commented out. If it's no longer needed, it should be removed entirely rather than left as a comment.Suggested fix
from app.models.llm import ( LLMCallRequest, NativeCompletionConfig, QueryParams, LLMCallResponse, LLMResponse, LLMOutput, Usage, - # KaapiLLMParams, KaapiCompletionConfig, )backend/app/services/llm/input_resolver.py (1)
88-96: Consider adding a streaming option for large audio files.The current implementation loads the entire response into memory with
response.content. For large audio files, this could cause memory issues. Consider usingstream=Trueand writing chunks to the temp file.Suggested streaming implementation
def resolve_audio_url(url: str) -> tuple[str, str | None]: """Fetch audio from URL and write to temp file. Returns (file_path, error).""" try: - response = requests.get(url, timeout=60) + response = requests.get(url, timeout=60, stream=True) response.raise_for_status() except requests.Timeout: return "", f"Timeout fetching audio from URL: {url}" except requests.HTTPError as e: return "", f"HTTP error fetching audio: {e.response.status_code}" except Exception as e: return "", f"Failed to fetch audio from URL: {str(e)}" content_type = response.headers.get("content-type", "audio/wav") ext = get_file_extension(content_type.split(";")[0].strip()) try: with tempfile.NamedTemporaryFile( suffix=ext, delete=False, prefix="audio_" ) as tmp: - tmp.write(response.content) + for chunk in response.iter_content(chunk_size=8192): + if chunk: + tmp.write(chunk) temp_path = tmp.name logger.info(f"[resolve_audio_url] Wrote audio to temp file: {temp_path}") return temp_path, None except Exception as e: return "", f"Failed to write fetched audio to temp file: {str(e)}"backend/app/tests/api/routes/configs/test_version.py (3)
565-570: Consider adding error message validation for consistency.The
test_create_version_cannot_change_type_from_text_to_stttest validates the error message content, but this test only checks the status code. Adding similar assertions would improve test coverage consistency.response = client.post( f"{settings.API_V1_STR}/configs/{config.id}/versions", headers={"X-API-KEY": user_api_key.key}, json=version_data, ) assert response.status_code == 400 + error_detail = response.json().get("error", "") + assert "cannot change config type" in error_detail.lower()
734-738: Unuseddbparameter flagged by static analysis.The
dbparameter is not used in this test function. If it's required for test fixture setup/teardown, consider adding a comment explaining its purpose. Otherwise, it can be removed.def test_create_config_with_kaapi_provider_success( - db: Session, client: TestClient, user_api_key: TestAuthContext, ) -> None:
477-478: Consider moving repeated imports to module level.
KaapiCompletionConfigis imported locally in multiple test functions. Moving it to the module-level imports (alongsideNativeCompletionConfigat line 14) would reduce duplication.backend/app/tests/utils/test_data.py (1)
321-337: Move imports to module level.
selectandand_from sqlmodel, andConfigVersionare imported inside the function, butConfigVersionis already imported at line 19. Consider consolidating these imports at the module level.+from sqlmodel import Session, select, and_ # ... at module level def create_test_version(...): if config_blob is None: - # Fetch the latest version to maintain type consistency - from sqlmodel import select, and_ - from app.models import ConfigVersion - stmt = ( select(ConfigVersion)backend/app/services/llm/providers/registry.py (1)
1-2: Remove unnecessary imports for production code.
osanddotenvare only used in the__main__testing block. If the testing code is removed (as suggested below), these imports should also be removed.backend/app/services/llm/providers/gai.py (1)
1-2: Remove unused import.
osis imported but never used in this file.import logging -import osbackend/app/crud/config/version.py (2)
170-190: Shallow copy may cause unintended mutation of nested structures.
base.copy()on Line 178 creates a shallow copy. Ifbasecontains nested dicts that are not inupdates, those nested dicts will be shared references. While the recursive merge handles overlapping keys correctly, any external mutation of the returned result could affect the originalbasedict.Consider using
copy.deepcopy()if the base dict may be reused or if nested structures need isolation.♻️ Suggested fix using deepcopy
+import copy + def _deep_merge( self, base: dict[str, Any], updates: dict[str, Any] ) -> dict[str, Any]: """ Deep merge two dictionaries. Values from 'updates' override values in 'base'. Nested dicts are merged recursively. """ - result = base.copy() + result = copy.deepcopy(base) for key, value in updates.items(): if ( key in result and isinstance(result[key], dict) and isinstance(value, dict) ): result[key] = self._deep_merge(result[key], value) else: result[key] = value return result
291-341: Consider reusing_get_latest_versionto reduce code duplication.The query logic in Lines 300-311 duplicates the query in
_get_latest_version(Lines 157-168). This creates maintenance burden and potential for divergence.♻️ Proposed refactor to reuse existing helper
def _validate_config_type_unchanged( self, version_create: ConfigVersionCreate ) -> None: """ Validate that the config type (text/stt/tts) in the new version matches the type from the latest existing version. Raises HTTPException if types don't match. """ - # Get the latest version - stmt = ( - select(ConfigVersion) - .where( - and_( - ConfigVersion.config_id == self.config_id, - ConfigVersion.deleted_at.is_(None), - ) - ) - .order_by(ConfigVersion.version.desc()) - .limit(1) - ) - latest_version = self.session.exec(stmt).first() + latest_version = self._get_latest_version() # If this is the first version, no validation needed if latest_version is None: returnbackend/app/crud/llm.py (3)
80-105: PotentialAttributeErrorwhen accessingparams.model.Line 102-105 uses
hasattrto check formodelattribute but assumescompletion_config.paramsexists and is accessible. Ifparamsis a dict (which it can be based onNativeCompletionConfig.params: dict[str, Any]),hasattr(completion_config.params, "model")will returnFalseeven when"model"is a key, and.get("model", "")should work. However, mixing attribute access with dict access creates confusion.♻️ Suggested simplification for clarity
- model = ( - completion_config.params.model - if hasattr(completion_config.params, "model") - else completion_config.params.get("model", "") - ) + # params is always a dict for both Native and Kaapi configs + model = completion_config.params.get("model", "")
99-100: Type ignore comment suggests potential type mismatch.The
# type: ignore[assignment]comment on Line 100 indicates the provider type doesn't match the expectedLiteral["openai", "google", "anthropic"]. The comment mentions provider is "guaranteed to be normalized" but this isn't enforced at the type level. Consider adding a runtime assertion or documenting the contract more explicitly.
200-229: Add type hint for return value onget_llm_calls_by_job_id.The function is missing a return type annotation per the coding guidelines requiring type hints on all function parameters and return values.
♻️ Proposed fix
def get_llm_calls_by_job_id( session: Session, job_id: UUID, -) -> list[LlmCall]: +) -> list[LlmCall]:Actually, looking again, the return type is already specified. The function signatures look complete.
LGTM!
Query functions are well-typed and properly filter out soft-deleted records.
backend/app/models/llm/request.py (1)
63-63: Consider usingX | Yunion syntax for consistency.Static analysis (UP007) suggests using
X | Yinstead ofUnion[...]for type annotations to align with modern Python 3.11+ syntax. This is optional but improves consistency with other type hints in the codebase.♻️ Proposed fix
-KaapiLLMParams = Union[TextLLMParams, STTLLMParams, TTSLLMParams] +KaapiLLMParams = TextLLMParams | STTLLMParams | TTSLLMParams # Discriminated union for query input types -QueryInput = Annotated[ - Union[TextInput, AudioBase64Input, AudioUrlInput], - Field(discriminator="type"), -] +QueryInput = Annotated[ + TextInput | AudioBase64Input | AudioUrlInput, + Field(discriminator="type"), +]Also applies to: 87-90
Comment on lines
4
to
+11
| from sqlmodel import Field, SQLModel | ||
| from pydantic import Discriminator, model_validator, HttpUrl | ||
| from datetime import datetime | ||
| from app.core.util import now | ||
|
|
||
| import sqlalchemy as sa | ||
| from sqlalchemy.dialects.postgresql import JSONB | ||
| from sqlmodel import Field, SQLModel, Index, text |
There was a problem hiding this comment.
Remove duplicate imports flagged by static analysis.
Lines 4 and 11 both import Field and SQLModel from sqlmodel. This is flagged by Ruff (F811).
🐛 Proposed fix
from uuid import UUID, uuid4
-from sqlmodel import Field, SQLModel
from pydantic import Discriminator, model_validator, HttpUrl
from datetime import datetime
from app.core.util import now
import sqlalchemy as sa
from sqlalchemy.dialects.postgresql import JSONB
from sqlmodel import Field, SQLModel, Index, text🧰 Tools
🪛 Ruff (0.14.14)
11-11: Redefinition of unused Field from line 4: Field redefined here
Remove definition: Field
(F811)
11-11: Redefinition of unused SQLModel from line 4: SQLModel redefined here
Remove definition: SQLModel
(F811)
🤖 Prompt for AI Agents
In `@backend/app/models/llm/request.py` around lines 4 - 11, Remove the duplicate
import of Field and SQLModel: consolidate the two sqlmodel import lines into one
(e.g., replace the two occurrences of "from sqlmodel import Field, SQLModel"
with a single line that also includes Index and text if needed: "from sqlmodel
import Field, SQLModel, Index, text"), leaving other imports (pydantic,
datetime, sqlalchemy, JSONB, app.core.util) unchanged; ensure only one import
statement provides Field and SQLModel to fix the Ruff F811 duplicate-import
error.
Comment on lines
+313
to
+479
| class LlmCall(SQLModel, table=True): | ||
| """ | ||
| Database model for tracking LLM API call requests and responses. | ||
|
|
||
| Stores both request inputs and response outputs for traceability, | ||
| supporting multimodal inputs (text, audio, image) and various completion types. | ||
| """ | ||
|
|
||
| __tablename__ = "llm_call" | ||
| __table_args__ = ( | ||
| Index( | ||
| "idx_llm_call_job_id", | ||
| "job_id", | ||
| postgresql_where=text("deleted_at IS NULL"), | ||
| ), | ||
| Index( | ||
| "idx_llm_call_conversation_id", | ||
| "conversation_id", | ||
| postgresql_where=text("conversation_id IS NOT NULL AND deleted_at IS NULL"), | ||
| ), | ||
| ) | ||
|
|
||
| id: UUID = Field( | ||
| default_factory=uuid4, | ||
| primary_key=True, | ||
| sa_column_kwargs={"comment": "Unique identifier for the LLM call record"}, | ||
| ) | ||
|
|
||
| job_id: UUID = Field( | ||
| foreign_key="job.id", | ||
| nullable=False, | ||
| ondelete="CASCADE", | ||
| sa_column_kwargs={ | ||
| "comment": "Reference to the parent job (status tracked in job table)" | ||
| }, | ||
| ) | ||
|
|
||
| project_id: int = Field( | ||
| foreign_key="project.id", | ||
| nullable=False, | ||
| ondelete="CASCADE", | ||
| sa_column_kwargs={ | ||
| "comment": "Reference to the project this LLM call belongs to" | ||
| }, | ||
| ) | ||
|
|
||
| organization_id: int = Field( | ||
| foreign_key="organization.id", | ||
| nullable=False, | ||
| ondelete="CASCADE", | ||
| sa_column_kwargs={ | ||
| "comment": "Reference to the organization this LLM call belongs to" | ||
| }, | ||
| ) | ||
|
|
||
| # Request fields | ||
| input: str = Field( | ||
| ..., | ||
| sa_column_kwargs={ | ||
| "comment": "User input - text string, binary data, or file path for multimodal" | ||
| }, | ||
| ) | ||
|
|
||
| input_type: Literal["text", "audio", "image"] = Field( | ||
| ..., | ||
| sa_column=sa.Column( | ||
| sa.String, | ||
| nullable=False, | ||
| comment="Input type: text, audio, image", | ||
| ), | ||
| ) | ||
|
|
||
| output_type: Literal["text", "audio", "image"] | None = Field( | ||
| default=None, | ||
| sa_column=sa.Column( | ||
| sa.String, | ||
| nullable=True, | ||
| comment="Expected output type: text, audio, image", | ||
| ), | ||
| ) | ||
|
|
||
| # Provider and model info | ||
| provider: Literal["openai", "google", "anthropic"] = Field( | ||
| ..., | ||
| sa_column=sa.Column( | ||
| sa.String, | ||
| nullable=False, | ||
| comment="AI provider: openai, google, anthropic", | ||
| ), | ||
| ) | ||
|
|
||
| model: str = Field( | ||
| ..., | ||
| sa_column_kwargs={ | ||
| "comment": "Specific model used e.g. 'gpt-4o', 'gemini-2.5-pro'" | ||
| }, | ||
| ) | ||
|
|
||
| # Response fields | ||
| provider_response_id: str | None = Field( | ||
| default=None, | ||
| sa_column_kwargs={ | ||
| "comment": "Original response ID from the provider (e.g., OpenAI's response ID)" | ||
| }, | ||
| ) | ||
|
|
||
| content: dict[str, Any] | None = Field( | ||
| default=None, | ||
| sa_column=sa.Column( | ||
| JSONB, | ||
| nullable=True, | ||
| comment="Response content: {text: '...'}, {audio_bytes: '...'}, or {image: '...'}", | ||
| ), | ||
| ) | ||
|
|
||
| usage: dict[str, Any] | None = Field( | ||
| default=None, | ||
| sa_column=sa.Column( | ||
| JSONB, | ||
| nullable=True, | ||
| comment="Token usage: {input_tokens, output_tokens, reasoning_tokens}", | ||
| ), | ||
| ) | ||
|
|
||
| # Conversation tracking | ||
| conversation_id: str | None = Field( | ||
| default=None, | ||
| sa_column_kwargs={ | ||
| "comment": "Identifier linking this response to its conversation thread" | ||
| }, | ||
| ) | ||
|
|
||
| auto_create: bool | None = Field( | ||
| default=None, | ||
| sa_column_kwargs={ | ||
| "comment": "Whether to auto-create conversation if conversation_id doesn't exist (OpenAI specific)" | ||
| }, | ||
| ) | ||
|
|
||
| # Configuration - stores either {config_id, config_version} or {config_blob} | ||
| config: dict[str, Any] | None = Field( | ||
| default=None, | ||
| sa_column=sa.Column( | ||
| JSONB, | ||
| nullable=True, | ||
| comment="Configuration: {config_id, config_version} for stored config OR {config_blob} for ad-hoc config", | ||
| ), | ||
| ) | ||
|
|
||
| # Timestamps | ||
| created_at: datetime = Field( | ||
| default_factory=now, | ||
| nullable=False, | ||
| sa_column_kwargs={"comment": "Timestamp when the LLM call was created"}, | ||
| ) | ||
|
|
||
| updated_at: datetime = Field( | ||
| default_factory=now, | ||
| nullable=False, | ||
| sa_column_kwargs={"comment": "Timestamp when the LLM call was last updated"}, | ||
| ) | ||
|
|
||
| deleted_at: datetime | None = Field( | ||
| default=None, | ||
| nullable=True, | ||
| sa_column_kwargs={"comment": "Timestamp when the record was soft-deleted"}, | ||
| ) |
There was a problem hiding this comment.
updated_at field does not auto-update on record modification.
The updated_at field uses default_factory=now which sets the value only on creation. For proper audit tracking, this field should be updated whenever the record is modified. Consider adding sa_column_kwargs with onupdate or handling this in the CRUD layer.
🐛 Proposed fix using SQLAlchemy onupdate
updated_at: datetime = Field(
default_factory=now,
nullable=False,
- sa_column_kwargs={"comment": "Timestamp when the LLM call was last updated"},
+ sa_column_kwargs={
+ "comment": "Timestamp when the LLM call was last updated",
+ "onupdate": now,
+ },
)Alternatively, handle this in the update_llm_call_response function in backend/app/crud/llm.py:
if conversation_id is not None:
db_llm_call.conversation_id = conversation_id
+ db_llm_call.updated_at = now()
session.add(db_llm_call)🤖 Prompt for AI Agents
In `@backend/app/models/llm/request.py` around lines 313 - 479, The updated_at
field on the LlmCall model currently uses default_factory=now so it only sets at
creation; make it auto-update on modifications by adding an SQLAlchemy onupdate
to the Column (e.g., sa_column or sa_column_kwargs for updated_at with
onupdate=now) or, if you prefer application-level handling, ensure the
update_llm_call_response CRUD function (or any updater) sets updated_at = now()
before committing. Update the LlmCall.updated_at definition accordingly and/or
modify update_llm_call_response to assign now() on each update so updated_at
reflects the last modification.
Comment on lines
+86
to
+111
| def resolve_audio_url(url: str) -> tuple[str, str | None]: | ||
| """Fetch audio from URL and write to temp file. Returns (file_path, error).""" | ||
| try: | ||
| response = requests.get(url, timeout=60) | ||
| response.raise_for_status() | ||
| except requests.Timeout: | ||
| return "", f"Timeout fetching audio from URL: {url}" | ||
| except requests.HTTPError as e: | ||
| return "", f"HTTP error fetching audio: {e.response.status_code}" | ||
| except Exception as e: | ||
| return "", f"Failed to fetch audio from URL: {str(e)}" | ||
|
|
||
| content_type = response.headers.get("content-type", "audio/wav") | ||
| ext = get_file_extension(content_type.split(";")[0].strip()) | ||
|
|
||
| try: | ||
| with tempfile.NamedTemporaryFile( | ||
| suffix=ext, delete=False, prefix="audio_" | ||
| ) as tmp: | ||
| tmp.write(response.content) | ||
| temp_path = tmp.name | ||
|
|
||
| logger.info(f"[resolve_audio_url] Wrote audio to temp file: {temp_path}") | ||
| return temp_path, None | ||
| except Exception as e: | ||
| return "", f"Failed to write fetched audio to temp file: {str(e)}" |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Search for callers of resolve_audio_url and resolve_input to check for URL validation
rg -n "resolve_audio_url|resolve_input" --type=py -B5 -A5Repository: ProjectTech4DevAI/kaapi-backend
Length of output: 5840
🏁 Script executed:
#!/bin/bash
# Search for QueryInput and AudioUrlInput definitions
rg -n "class QueryInput|class AudioUrlInput|AudioUrlInput.*:" --type=py -A10Repository: ProjectTech4DevAI/kaapi-backend
Length of output: 2200
🏁 Script executed:
#!/bin/bash
# Check for validators on AudioUrlInput and search for URL validation logic
rg -n "validator|@field_validator|private_ip|ssrf|169.254|127.0.0.1" --type=py -B2 -A2 backend/app/models/llm/request.py
# Also search for any URL validation utilities
rg -n "validate.*url|check.*url|ssrf|private.*ip" --type=py -lRepository: ProjectTech4DevAI/kaapi-backend
Length of output: 1189
🏁 Script executed:
#!/bin/bash
# Check the SSRF test file
cat -n backend/app/tests/core/test_callback_ssrf.py
# Also search for any URL validation utilities in the codebase
rg -n "def.*validate.*url|def.*filter.*url|allowed.*url|blocked.*url" --type=py -B2 -A5Repository: ProjectTech4DevAI/kaapi-backend
Length of output: 15773
🏁 Script executed:
#!/bin/bash
# Get the full validate_callback_url and _is_private_ip implementations
sed -n '270,350p' backend/app/utils.py
# Confirm resolve_audio_url doesn't call any validation
rg -n "_is_private_ip|validate.*url" backend/app/services/llm/input_resolver.pyRepository: ProjectTech4DevAI/kaapi-backend
Length of output: 2374
Add SSRF protection to resolve_audio_url function.
The function fetches content from arbitrary user-controlled URLs without validation, allowing Server-Side Request Forgery attacks to access internal services or metadata endpoints (e.g., AWS metadata at 169.254.169.254). The codebase already implements _is_private_ip() and validate_callback_url() functions in app/utils.py for callback URL protection—reuse these for audio URL validation.
Required fixes:
- Call
validate_callback_url()on the URL before fetching - Or implement a less restrictive check (HTTPS-only + block private/link-local IP ranges)
- Disable redirects in the
requests.get()call to prevent redirect-based SSRF
🤖 Prompt for AI Agents
In `@backend/app/services/llm/input_resolver.py` around lines 86 - 111, The
resolve_audio_url function currently fetches arbitrary URLs; before calling
requests.get in resolve_audio_url, validate the input URL by reusing
validate_callback_url(url) (or at minimum enforce scheme == "https" and use
_is_private_ip to reject private/link-local IPs) and return an error string on
validation failure; also call requests.get with allow_redirects=False to disable
redirects and keep the existing timeout; keep existing temp file write logic
(references: resolve_audio_url, validate_callback_url, _is_private_ip,
get_file_extension).
Comment on lines
+160
to
+178
|
|
||
| # Debug: Try to fetch the job first | ||
| logger.info(f"[execute_job] Attempting to fetch job | job_id={job_id}") | ||
| job = session.get(Job, job_id) | ||
| if not job: | ||
| # Log all jobs to see what's in the database | ||
| from sqlmodel import select | ||
|
|
||
| all_jobs = session.exec( | ||
| select(Job).order_by(Job.created_at.desc()).limit(5) | ||
| ).all() | ||
| logger.error( | ||
| f"[execute_job] Job not found! | job_id={job_id} | " | ||
| f"Recent jobs in DB: {[(j.id, j.status) for j in all_jobs]}" | ||
| ) | ||
| else: | ||
| logger.info( | ||
| f"[execute_job] Found job | job_id={job_id}, status={job.status}" | ||
| ) |
There was a problem hiding this comment.
Remove debug code before merging.
This block appears to be debugging code that queries all recent jobs when the expected job isn't found. It adds unnecessary database queries in production and includes an inline import. Consider removing this or converting it to a proper diagnostic utility if this scenario needs monitoring.
🔧 Suggested removal
job_crud = JobCrud(session=session)
-
- # Debug: Try to fetch the job first
- logger.info(f"[execute_job] Attempting to fetch job | job_id={job_id}")
- job = session.get(Job, job_id)
- if not job:
- # Log all jobs to see what's in the database
- from sqlmodel import select
-
- all_jobs = session.exec(
- select(Job).order_by(Job.created_at.desc()).limit(5)
- ).all()
- logger.error(
- f"[execute_job] Job not found! | job_id={job_id} | "
- f"Recent jobs in DB: {[(j.id, j.status) for j in all_jobs]}"
- )
- else:
- logger.info(
- f"[execute_job] Found job | job_id={job_id}, status={job.status}"
- )
-
job_crud.update(🤖 Prompt for AI Agents
In `@backend/app/services/llm/jobs.py` around lines 160 - 178, Remove the
temporary debug block in execute_job that performs an inline import of select
and queries recent jobs when session.get(Job, job_id) returns None; delete the
extra session.exec(...) query, the inline "from sqlmodel import select" import,
and the verbose logger.error that prints recent jobs, leaving only the initial
logger.info that attempts to fetch the job and the existing logger.error (or add
a concise logger.error) for the missing Job; ensure any needed diagnostics are
moved to a dedicated utility function (e.g., a new diagnostic helper) rather
than inline in execute_job.
| completion_config, warnings = transform_kaapi_config_to_native( | ||
| completion_config | ||
| ) | ||
| print(f"The completion_config transformed is {completion_config}") |
There was a problem hiding this comment.
Replace print with logger call.
Debug output should use the logger for consistency and proper log level control. As per coding guidelines, log messages should be prefixed with the function name.
- print(f"The completion_config transformed is {completion_config}")
+ logger.debug(f"[execute_job] Transformed completion_config: {completion_config}")📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| print(f"The completion_config transformed is {completion_config}") | |
| logger.debug(f"[execute_job] Transformed completion_config: {completion_config}") |
🤖 Prompt for AI Agents
In `@backend/app/services/llm/jobs.py` at line 212, Replace the print call
printing completion_config with a logger call using the module logger (e.g.,
logger.debug or logger.info) instead of print; log the same message text but
prefixed with the current function name and include the completion_config
variable for context (in backend/app/services/llm/jobs.py, at the location where
completion_config is printed), ensuring you use the existing module logger (or
create one via logging.getLogger(__name__)) and the appropriate log level rather
than print.
Comment on lines
+66
to
70
| # e.g "openai-native" -> "openai", "claude-native" -> "claude" | ||
| credential_provider = provider_type.replace("-native", "") | ||
|
|
||
| # e.g., "openai-native" → "openai", "claude-native" → "claude" | ||
| credential_provider = provider_type.replace("-native", "") |
There was a problem hiding this comment.
Remove duplicate variable assignment.
credential_provider is assigned twice with identical logic. Remove the duplicate.
provider_class = LLMProvider.get_provider_class(provider_type)
# e.g "openai-native" -> "openai", "claude-native" -> "claude"
credential_provider = provider_type.replace("-native", "")
- # e.g., "openai-native" → "openai", "claude-native" → "claude"
- credential_provider = provider_type.replace("-native", "")
-
credentials = get_provider_credential(📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| # e.g "openai-native" -> "openai", "claude-native" -> "claude" | |
| credential_provider = provider_type.replace("-native", "") | |
| # e.g., "openai-native" → "openai", "claude-native" → "claude" | |
| credential_provider = provider_type.replace("-native", "") | |
| provider_class = LLMProvider.get_provider_class(provider_type) | |
| # e.g "openai-native" -> "openai", "claude-native" -> "claude" | |
| credential_provider = provider_type.replace("-native", "") | |
| credentials = get_provider_credential( |
🤖 Prompt for AI Agents
In `@backend/app/services/llm/providers/registry.py` around lines 66 - 70, There
is a duplicated assignment of credential_provider from provider_type using
replace("-native", ""); remove the redundant line so credential_provider is
assigned only once (keep the first or the clearer occurrence) and ensure any
surrounding comments remain correct; locate the duplicate by searching for the
variable name credential_provider and the expression
provider_type.replace("-native", "") in registry.py and delete the extra
assignment.
Comment on lines
+92
to
+135
| # ad hoc testing code | ||
| if __name__ == "__main__": | ||
| # 1. Simulate environment/credentials | ||
| GEMINI_KEY = os.getenv("GEMINI_API_KEY") | ||
| if not GEMINI_KEY: | ||
| print("Set GEMINI_API_KEY environment variable first.") | ||
| exit(1) | ||
|
|
||
| # This dictionary mimics what get_provider_credential would return from the DB | ||
| mock_credentials = {"api_key": GEMINI_KEY} | ||
|
|
||
| # 2. Idiomatic Initialization via Registry | ||
| provider_type = "google-native" | ||
| # provider_type=LLMProvider.get_provider_class(provider_type="GOOGLE-NATIVE") | ||
|
|
||
| print(f"Initializing provider: {provider_type}...") | ||
|
|
||
| # This block mimics the core logic of your get_llm_provider function | ||
| ProviderClass = LLMProvider.get_provider_class(provider_type) | ||
| client = ProviderClass.create_client(credentials=mock_credentials) | ||
| instance = ProviderClass(client=client) | ||
|
|
||
| # 3. Setup Config and Query | ||
| test_config = NativeCompletionConfig( | ||
| provider="google-native", | ||
| type="stt", | ||
| params={ | ||
| "model": "gemini-2.5-pro", | ||
| "instructions": "Please transcribe this audio accurately.", | ||
| }, | ||
| ) | ||
|
|
||
| test_query = QueryParams( | ||
| input="/Users/prajna/Desktop/personal/projects/software/Syspin_Hackathon_api_server/wav_files/1253534463206645.wav" # Ensure this file exists in your directory | ||
| ) | ||
|
|
||
| # 4. Execution | ||
| print("Executing STT...") | ||
| result, error = instance.execute(completion_config=test_config, query=test_query) | ||
|
|
||
| return provider_class(client=client) | ||
| if error: | ||
| print(f"Error: {error}") | ||
| else: | ||
| print(f"Result: {result}") |
There was a problem hiding this comment.
Remove ad hoc testing code from production module.
This __main__ block contains:
- Hardcoded local file paths (line 125) that won't work for other developers
- Outdated
execute()call signature missing theresolved_inputparameter (line 130) - Testing logic that belongs in a dedicated test file
Consider moving this to backend/app/tests/services/llm/providers/test_gai.py or removing it entirely.
🤖 Prompt for AI Agents
In `@backend/app/services/llm/providers/registry.py` around lines 92 - 135, Remove
the ad-hoc "__main__" test block from registry.py; the block contains hardcoded
paths and an outdated call signature. Extract the logic that uses
LLMProvider.get_provider_class, ProviderClass.create_client,
NativeCompletionConfig, QueryParams and instance.execute into a proper test
under backend/app/tests/services/llm/providers/test_gai.py (or delete it),
update the execute invocation to include the resolved_input parameter to match
the current signature, and ensure any credential/env handling is mocked rather
than reading real env vars or local file paths.
Comment on lines
+269
to
+271
| # Should not find with wrong project | ||
| fetched_wrong = get_llm_call_by_id(db, created.id, project_id=99999) | ||
| assert fetched_wrong is None |
There was a problem hiding this comment.
Type mismatch: project_id should be UUID, not int.
The test uses project_id=99999 (an integer), but based on other usages in the file (e.g., test_project.id), project_id appears to be a UUID type. This could cause a type error or unexpected behavior.
Suggested fix
# Should not find with wrong project
- fetched_wrong = get_llm_call_by_id(db, created.id, project_id=99999)
+ fetched_wrong = get_llm_call_by_id(db, created.id, project_id=uuid4())
assert fetched_wrong is None📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| # Should not find with wrong project | |
| fetched_wrong = get_llm_call_by_id(db, created.id, project_id=99999) | |
| assert fetched_wrong is None | |
| # Should not find with wrong project | |
| fetched_wrong = get_llm_call_by_id(db, created.id, project_id=uuid4()) | |
| assert fetched_wrong is None |
🤖 Prompt for AI Agents
In `@backend/app/tests/crud/test_llm.py` around lines 269 - 271, The test passes
an integer literal 99999 as project_id to get_llm_call_by_id but project_id is a
UUID; change the test to pass a UUID that will not match the created LLM call
(e.g., generate a new UUID via uuid.uuid4() or use a different fixture UUID)
instead of 99999 so the call uses the correct type; update the assertion code
around get_llm_call_by_id(db, created.id, project_id=...) and ensure
imports/fixtures provide a UUID value.
Comment on lines
+1
to
+317
| # """ | ||
| # Unit tests for LLM parameter mapping functions. | ||
|
|
||
| from app.models.llm import KaapiLLMParams, KaapiCompletionConfig, NativeCompletionConfig | ||
| from app.services.llm.mappers import ( | ||
| map_kaapi_to_openai_params, | ||
| transform_kaapi_config_to_native, | ||
| ) | ||
|
|
||
|
|
||
| class TestMapKaapiToOpenAIParams: | ||
| """Test cases for map_kaapi_to_openai_params function.""" | ||
|
|
||
| def test_basic_model_mapping(self): | ||
| """Test basic model parameter mapping.""" | ||
| kaapi_params = KaapiLLMParams(model="gpt-4o") | ||
|
|
||
| result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| assert result == {"model": "gpt-4o"} | ||
| assert warnings == [] | ||
|
|
||
| def test_instructions_mapping(self): | ||
| """Test instructions parameter mapping.""" | ||
| kaapi_params = KaapiLLMParams( | ||
| model="gpt-4", | ||
| instructions="You are a helpful assistant.", | ||
| ) | ||
|
|
||
| result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| assert result["model"] == "gpt-4" | ||
| assert result["instructions"] == "You are a helpful assistant." | ||
| assert warnings == [] | ||
|
|
||
| def test_temperature_mapping(self): | ||
| """Test temperature parameter mapping for non-reasoning models.""" | ||
| kaapi_params = KaapiLLMParams( | ||
| model="gpt-4", | ||
| temperature=0.7, | ||
| ) | ||
|
|
||
| result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| assert result["model"] == "gpt-4" | ||
| assert result["temperature"] == 0.7 | ||
| assert warnings == [] | ||
|
|
||
| def test_temperature_zero_mapping(self): | ||
| """Test that temperature=0 is correctly mapped (edge case).""" | ||
| kaapi_params = KaapiLLMParams( | ||
| model="gpt-4", | ||
| temperature=0.0, | ||
| ) | ||
|
|
||
| result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| assert result["temperature"] == 0.0 | ||
| assert warnings == [] | ||
|
|
||
| def test_reasoning_mapping_for_reasoning_models(self): | ||
| """Test reasoning parameter mapping to OpenAI format for reasoning-capable models.""" | ||
| kaapi_params = KaapiLLMParams( | ||
| model="o1", | ||
| reasoning="high", | ||
| ) | ||
|
|
||
| result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| assert result["model"] == "o1" | ||
| assert result["reasoning"] == {"effort": "high"} | ||
| assert warnings == [] | ||
|
|
||
| def test_knowledge_base_ids_mapping(self): | ||
| """Test knowledge_base_ids mapping to OpenAI tools format.""" | ||
| kaapi_params = KaapiLLMParams( | ||
| model="gpt-4", | ||
| knowledge_base_ids=["vs_abc123", "vs_def456"], | ||
| ) | ||
|
|
||
| result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| assert result["model"] == "gpt-4" | ||
| assert "tools" in result | ||
| assert len(result["tools"]) == 1 | ||
| assert result["tools"][0]["type"] == "file_search" | ||
| assert result["tools"][0]["vector_store_ids"] == ["vs_abc123", "vs_def456"] | ||
| assert result["tools"][0]["max_num_results"] == 20 # default | ||
| assert warnings == [] | ||
|
|
||
| def test_knowledge_base_with_max_num_results(self): | ||
| """Test knowledge_base_ids with custom max_num_results.""" | ||
| kaapi_params = KaapiLLMParams( | ||
| model="gpt-4", | ||
| knowledge_base_ids=["vs_abc123"], | ||
| max_num_results=50, | ||
| ) | ||
|
|
||
| result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| assert result["tools"][0]["max_num_results"] == 50 | ||
| assert warnings == [] | ||
|
|
||
| def test_complete_parameter_mapping(self): | ||
| """Test mapping all compatible parameters together.""" | ||
| kaapi_params = KaapiLLMParams( | ||
| model="gpt-4o", | ||
| instructions="You are an expert assistant.", | ||
| temperature=0.8, | ||
| knowledge_base_ids=["vs_123"], | ||
| max_num_results=30, | ||
| ) | ||
|
|
||
| result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| assert result["model"] == "gpt-4o" | ||
| assert result["instructions"] == "You are an expert assistant." | ||
| assert result["temperature"] == 0.8 | ||
| assert result["tools"][0]["type"] == "file_search" | ||
| assert result["tools"][0]["vector_store_ids"] == ["vs_123"] | ||
| assert result["tools"][0]["max_num_results"] == 30 | ||
| assert warnings == [] | ||
|
|
||
| def test_reasoning_suppressed_for_non_reasoning_models(self): | ||
| """Test that reasoning is suppressed with warning for non-reasoning models.""" | ||
| kaapi_params = KaapiLLMParams( | ||
| model="gpt-4", | ||
| reasoning="high", | ||
| ) | ||
|
|
||
| result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| assert result["model"] == "gpt-4" | ||
| assert "reasoning" not in result | ||
| assert len(warnings) == 1 | ||
| assert "reasoning" in warnings[0].lower() | ||
| assert "does not support reasoning" in warnings[0] | ||
|
|
||
| def test_temperature_suppressed_for_reasoning_models(self): | ||
| """Test that temperature is suppressed with warning for reasoning models when reasoning is set.""" | ||
| kaapi_params = KaapiLLMParams( | ||
| model="o1", | ||
| temperature=0.7, | ||
| reasoning="high", | ||
| ) | ||
|
|
||
| result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| assert result["model"] == "o1" | ||
| assert result["reasoning"] == {"effort": "high"} | ||
| assert "temperature" not in result | ||
| assert len(warnings) == 1 | ||
| assert "temperature" in warnings[0].lower() | ||
| assert "suppressed" in warnings[0] | ||
|
|
||
| def test_temperature_without_reasoning_for_reasoning_models(self): | ||
| """Test that temperature is suppressed for reasoning models even without explicit reasoning parameter.""" | ||
| kaapi_params = KaapiLLMParams( | ||
| model="o1", | ||
| temperature=0.7, | ||
| ) | ||
|
|
||
| result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| assert result["model"] == "o1" | ||
| assert "temperature" not in result | ||
| assert "reasoning" not in result | ||
| assert len(warnings) == 1 | ||
| assert "temperature" in warnings[0].lower() | ||
| assert "suppressed" in warnings[0] | ||
|
|
||
| def test_minimal_params(self): | ||
| """Test mapping with minimal parameters (only model).""" | ||
| kaapi_params = KaapiLLMParams(model="gpt-4") | ||
|
|
||
| result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| assert result == {"model": "gpt-4"} | ||
| assert warnings == [] | ||
|
|
||
| def test_only_knowledge_base_ids(self): | ||
| """Test mapping with only knowledge_base_ids and model.""" | ||
| kaapi_params = KaapiLLMParams( | ||
| model="gpt-4", | ||
| knowledge_base_ids=["vs_xyz"], | ||
| ) | ||
|
|
||
| result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| assert result["model"] == "gpt-4" | ||
| assert "tools" in result | ||
| assert result["tools"][0]["vector_store_ids"] == ["vs_xyz"] | ||
| assert warnings == [] | ||
|
|
||
|
|
||
| class TestTransformKaapiConfigToNative: | ||
| """Test cases for transform_kaapi_config_to_native function.""" | ||
|
|
||
| def test_transform_openai_config(self): | ||
| """Test transformation of Kaapi OpenAI config to native format.""" | ||
| kaapi_config = KaapiCompletionConfig( | ||
| provider="openai", | ||
| params=KaapiLLMParams( | ||
| model="gpt-4", | ||
| temperature=0.7, | ||
| ), | ||
| ) | ||
|
|
||
| result, warnings = transform_kaapi_config_to_native(kaapi_config) | ||
|
|
||
| assert isinstance(result, NativeCompletionConfig) | ||
| assert result.provider == "openai-native" | ||
| assert result.params["model"] == "gpt-4" | ||
| assert result.params["temperature"] == 0.7 | ||
| assert warnings == [] | ||
|
|
||
| def test_transform_with_all_params(self): | ||
| """Test transformation with all Kaapi parameters.""" | ||
| kaapi_config = KaapiCompletionConfig( | ||
| provider="openai", | ||
| params=KaapiLLMParams( | ||
| model="gpt-4o", | ||
| instructions="System prompt here", | ||
| temperature=0.5, | ||
| knowledge_base_ids=["vs_abc"], | ||
| max_num_results=25, | ||
| ), | ||
| ) | ||
|
|
||
| result, warnings = transform_kaapi_config_to_native(kaapi_config) | ||
|
|
||
| assert result.provider == "openai-native" | ||
| assert result.params["model"] == "gpt-4o" | ||
| assert result.params["instructions"] == "System prompt here" | ||
| assert result.params["temperature"] == 0.5 | ||
| assert result.params["tools"][0]["type"] == "file_search" | ||
| assert result.params["tools"][0]["max_num_results"] == 25 | ||
| assert warnings == [] | ||
|
|
||
| def test_transform_with_reasoning(self): | ||
| """Test transformation with reasoning parameter for reasoning-capable models.""" | ||
| kaapi_config = KaapiCompletionConfig( | ||
| provider="openai", | ||
| params=KaapiLLMParams( | ||
| model="o1", | ||
| reasoning="medium", | ||
| ), | ||
| ) | ||
|
|
||
| result, warnings = transform_kaapi_config_to_native(kaapi_config) | ||
|
|
||
| assert result.provider == "openai-native" | ||
| assert result.params["model"] == "o1" | ||
| assert result.params["reasoning"] == {"effort": "medium"} | ||
| assert warnings == [] | ||
|
|
||
| def test_transform_with_both_temperature_and_reasoning(self): | ||
| """Test that transformation handles temperature + reasoning intelligently for reasoning models.""" | ||
| kaapi_config = KaapiCompletionConfig( | ||
| provider="openai", | ||
| params=KaapiLLMParams( | ||
| model="o1", | ||
| temperature=0.7, | ||
| reasoning="high", | ||
| ), | ||
| ) | ||
|
|
||
| result, warnings = transform_kaapi_config_to_native(kaapi_config) | ||
|
|
||
| assert result.provider == "openai-native" | ||
| assert result.params["model"] == "o1" | ||
| assert result.params["reasoning"] == {"effort": "high"} | ||
| assert "temperature" not in result.params | ||
| assert len(warnings) == 1 | ||
| assert "temperature" in warnings[0].lower() | ||
| assert "suppressed" in warnings[0] | ||
|
|
||
| def test_unsupported_provider_raises_error(self): | ||
| """Test that unsupported providers raise ValueError.""" | ||
| # Note: This would require modifying KaapiCompletionConfig to accept other providers | ||
| # For now, this tests the error handling in the mapper | ||
| # We'll create a mock config that bypasses validation | ||
| from unittest.mock import MagicMock | ||
|
|
||
| mock_config = MagicMock() | ||
| mock_config.provider = "unsupported-provider" | ||
| mock_config.params = KaapiLLMParams(model="some-model") | ||
|
|
||
| with pytest.raises(ValueError) as exc_info: | ||
| transform_kaapi_config_to_native(mock_config) | ||
|
|
||
| assert "Unsupported provider" in str(exc_info.value) | ||
|
|
||
| def test_transform_preserves_param_structure(self): | ||
| """Test that transformation correctly structures nested parameters.""" | ||
| kaapi_config = KaapiCompletionConfig( | ||
| provider="openai", | ||
| params=KaapiLLMParams( | ||
| model="gpt-4", | ||
| knowledge_base_ids=["vs_1", "vs_2", "vs_3"], | ||
| max_num_results=15, | ||
| ), | ||
| ) | ||
|
|
||
| result, warnings = transform_kaapi_config_to_native(kaapi_config) | ||
|
|
||
| # Verify the nested structure is correct | ||
| assert isinstance(result.params["tools"], list) | ||
| assert isinstance(result.params["tools"][0], dict) | ||
| assert isinstance(result.params["tools"][0]["vector_store_ids"], list) | ||
| assert len(result.params["tools"][0]["vector_store_ids"]) == 3 | ||
| assert warnings == [] | ||
| # Tests the transformation of Kaapi-abstracted parameters to provider-native formats. | ||
| # """ | ||
|
|
||
| # import pytest | ||
|
|
||
| # from app.models.llm import KaapiLLMParams, KaapiCompletionConfig, NativeCompletionConfig | ||
| # from app.services.llm.mappers import ( | ||
| # map_kaapi_to_openai_params, | ||
| # transform_kaapi_config_to_native, | ||
| # ) | ||
|
|
||
|
|
||
| # class TestMapKaapiToOpenAIParams: | ||
| # """Test cases for map_kaapi_to_openai_params function.""" | ||
|
|
||
| # def test_basic_model_mapping(self): | ||
| # """Test basic model parameter mapping.""" | ||
| # kaapi_params = KaapiLLMParams(model="gpt-4o") | ||
|
|
||
| # result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| # assert result == {"model": "gpt-4o"} | ||
| # assert warnings == [] | ||
|
|
||
| # def test_instructions_mapping(self): | ||
| # """Test instructions parameter mapping.""" | ||
| # kaapi_params = KaapiLLMParams( | ||
| # model="gpt-4", | ||
| # instructions="You are a helpful assistant.", | ||
| # ) | ||
|
|
||
| # result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| # assert result["model"] == "gpt-4" | ||
| # assert result["instructions"] == "You are a helpful assistant." | ||
| # assert warnings == [] | ||
|
|
||
| # def test_temperature_mapping(self): | ||
| # """Test temperature parameter mapping for non-reasoning models.""" | ||
| # kaapi_params = KaapiLLMParams( | ||
| # model="gpt-4", | ||
| # temperature=0.7, | ||
| # ) | ||
|
|
||
| # result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| # assert result["model"] == "gpt-4" | ||
| # assert result["temperature"] == 0.7 | ||
| # assert warnings == [] | ||
|
|
||
| # def test_temperature_zero_mapping(self): | ||
| # """Test that temperature=0 is correctly mapped (edge case).""" | ||
| # kaapi_params = KaapiLLMParams( | ||
| # model="gpt-4", | ||
| # temperature=0.0, | ||
| # ) | ||
|
|
||
| # result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| # assert result["temperature"] == 0.0 | ||
| # assert warnings == [] | ||
|
|
||
| # def test_reasoning_mapping_for_reasoning_models(self): | ||
| # """Test reasoning parameter mapping to OpenAI format for reasoning-capable models.""" | ||
| # kaapi_params = KaapiLLMParams( | ||
| # model="o1", | ||
| # reasoning="high", | ||
| # ) | ||
|
|
||
| # result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| # assert result["model"] == "o1" | ||
| # assert result["reasoning"] == {"effort": "high"} | ||
| # assert warnings == [] | ||
|
|
||
| # def test_knowledge_base_ids_mapping(self): | ||
| # """Test knowledge_base_ids mapping to OpenAI tools format.""" | ||
| # kaapi_params = KaapiLLMParams( | ||
| # model="gpt-4", | ||
| # knowledge_base_ids=["vs_abc123", "vs_def456"], | ||
| # ) | ||
|
|
||
| # result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| # assert result["model"] == "gpt-4" | ||
| # assert "tools" in result | ||
| # assert len(result["tools"]) == 1 | ||
| # assert result["tools"][0]["type"] == "file_search" | ||
| # assert result["tools"][0]["vector_store_ids"] == ["vs_abc123", "vs_def456"] | ||
| # assert result["tools"][0]["max_num_results"] == 20 # default | ||
| # assert warnings == [] | ||
|
|
||
| # def test_knowledge_base_with_max_num_results(self): | ||
| # """Test knowledge_base_ids with custom max_num_results.""" | ||
| # kaapi_params = KaapiLLMParams( | ||
| # model="gpt-4", | ||
| # knowledge_base_ids=["vs_abc123"], | ||
| # max_num_results=50, | ||
| # ) | ||
|
|
||
| # result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| # assert result["tools"][0]["max_num_results"] == 50 | ||
| # assert warnings == [] | ||
|
|
||
| # def test_complete_parameter_mapping(self): | ||
| # """Test mapping all compatible parameters together.""" | ||
| # kaapi_params = KaapiLLMParams( | ||
| # model="gpt-4o", | ||
| # instructions="You are an expert assistant.", | ||
| # temperature=0.8, | ||
| # knowledge_base_ids=["vs_123"], | ||
| # max_num_results=30, | ||
| # ) | ||
|
|
||
| # result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| # assert result["model"] == "gpt-4o" | ||
| # assert result["instructions"] == "You are an expert assistant." | ||
| # assert result["temperature"] == 0.8 | ||
| # assert result["tools"][0]["type"] == "file_search" | ||
| # assert result["tools"][0]["vector_store_ids"] == ["vs_123"] | ||
| # assert result["tools"][0]["max_num_results"] == 30 | ||
| # assert warnings == [] | ||
|
|
||
| # def test_reasoning_suppressed_for_non_reasoning_models(self): | ||
| # """Test that reasoning is suppressed with warning for non-reasoning models.""" | ||
| # kaapi_params = KaapiLLMParams( | ||
| # model="gpt-4", | ||
| # reasoning="high", | ||
| # ) | ||
|
|
||
| # result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| # assert result["model"] == "gpt-4" | ||
| # assert "reasoning" not in result | ||
| # assert len(warnings) == 1 | ||
| # assert "reasoning" in warnings[0].lower() | ||
| # assert "does not support reasoning" in warnings[0] | ||
|
|
||
| # def test_temperature_suppressed_for_reasoning_models(self): | ||
| # """Test that temperature is suppressed with warning for reasoning models when reasoning is set.""" | ||
| # kaapi_params = KaapiLLMParams( | ||
| # model="o1", | ||
| # temperature=0.7, | ||
| # reasoning="high", | ||
| # ) | ||
|
|
||
| # result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| # assert result["model"] == "o1" | ||
| # assert result["reasoning"] == {"effort": "high"} | ||
| # assert "temperature" not in result | ||
| # assert len(warnings) == 1 | ||
| # assert "temperature" in warnings[0].lower() | ||
| # assert "suppressed" in warnings[0] | ||
|
|
||
| # def test_temperature_without_reasoning_for_reasoning_models(self): | ||
| # """Test that temperature is suppressed for reasoning models even without explicit reasoning parameter.""" | ||
| # kaapi_params = KaapiLLMParams( | ||
| # model="o1", | ||
| # temperature=0.7, | ||
| # ) | ||
|
|
||
| # result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| # assert result["model"] == "o1" | ||
| # assert "temperature" not in result | ||
| # assert "reasoning" not in result | ||
| # assert len(warnings) == 1 | ||
| # assert "temperature" in warnings[0].lower() | ||
| # assert "suppressed" in warnings[0] | ||
|
|
||
| # def test_minimal_params(self): | ||
| # """Test mapping with minimal parameters (only model).""" | ||
| # kaapi_params = KaapiLLMParams(model="gpt-4") | ||
|
|
||
| # result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| # assert result == {"model": "gpt-4"} | ||
| # assert warnings == [] | ||
|
|
||
| # def test_only_knowledge_base_ids(self): | ||
| # """Test mapping with only knowledge_base_ids and model.""" | ||
| # kaapi_params = KaapiLLMParams( | ||
| # model="gpt-4", | ||
| # knowledge_base_ids=["vs_xyz"], | ||
| # ) | ||
|
|
||
| # result, warnings = map_kaapi_to_openai_params(kaapi_params) | ||
|
|
||
| # assert result["model"] == "gpt-4" | ||
| # assert "tools" in result | ||
| # assert result["tools"][0]["vector_store_ids"] == ["vs_xyz"] | ||
| # assert warnings == [] | ||
|
|
||
|
|
||
| # class TestTransformKaapiConfigToNative: | ||
| # """Test cases for transform_kaapi_config_to_native function.""" | ||
|
|
||
| # def test_transform_openai_config(self): | ||
| # """Test transformation of Kaapi OpenAI config to native format.""" | ||
| # kaapi_config = KaapiCompletionConfig( | ||
| # provider="openai", | ||
| # params=KaapiLLMParams( | ||
| # model="gpt-4", | ||
| # temperature=0.7, | ||
| # ), | ||
| # ) | ||
|
|
||
| # result, warnings = transform_kaapi_config_to_native(kaapi_config) | ||
|
|
||
| # assert isinstance(result, NativeCompletionConfig) | ||
| # assert result.provider == "openai-native" | ||
| # assert result.params["model"] == "gpt-4" | ||
| # assert result.params["temperature"] == 0.7 | ||
| # assert warnings == [] | ||
|
|
||
| # def test_transform_with_all_params(self): | ||
| # """Test transformation with all Kaapi parameters.""" | ||
| # kaapi_config = KaapiCompletionConfig( | ||
| # provider="openai", | ||
| # params=KaapiLLMParams( | ||
| # model="gpt-4o", | ||
| # instructions="System prompt here", | ||
| # temperature=0.5, | ||
| # knowledge_base_ids=["vs_abc"], | ||
| # max_num_results=25, | ||
| # ), | ||
| # ) | ||
|
|
||
| # result, warnings = transform_kaapi_config_to_native(kaapi_config) | ||
|
|
||
| # assert result.provider == "openai-native" | ||
| # assert result.params["model"] == "gpt-4o" | ||
| # assert result.params["instructions"] == "System prompt here" | ||
| # assert result.params["temperature"] == 0.5 | ||
| # assert result.params["tools"][0]["type"] == "file_search" | ||
| # assert result.params["tools"][0]["max_num_results"] == 25 | ||
| # assert warnings == [] | ||
|
|
||
| # def test_transform_with_reasoning(self): | ||
| # """Test transformation with reasoning parameter for reasoning-capable models.""" | ||
| # kaapi_config = KaapiCompletionConfig( | ||
| # provider="openai", | ||
| # params=KaapiLLMParams( | ||
| # model="o1", | ||
| # reasoning="medium", | ||
| # ), | ||
| # ) | ||
|
|
||
| # result, warnings = transform_kaapi_config_to_native(kaapi_config) | ||
|
|
||
| # assert result.provider == "openai-native" | ||
| # assert result.params["model"] == "o1" | ||
| # assert result.params["reasoning"] == {"effort": "medium"} | ||
| # assert warnings == [] | ||
|
|
||
| # def test_transform_with_both_temperature_and_reasoning(self): | ||
| # """Test that transformation handles temperature + reasoning intelligently for reasoning models.""" | ||
| # kaapi_config = KaapiCompletionConfig( | ||
| # provider="openai", | ||
| # params=KaapiLLMParams( | ||
| # model="o1", | ||
| # temperature=0.7, | ||
| # reasoning="high", | ||
| # ), | ||
| # ) | ||
|
|
||
| # result, warnings = transform_kaapi_config_to_native(kaapi_config) | ||
|
|
||
| # assert result.provider == "openai-native" | ||
| # assert result.params["model"] == "o1" | ||
| # assert result.params["reasoning"] == {"effort": "high"} | ||
| # assert "temperature" not in result.params | ||
| # assert len(warnings) == 1 | ||
| # assert "temperature" in warnings[0].lower() | ||
| # assert "suppressed" in warnings[0] | ||
|
|
||
| # def test_unsupported_provider_raises_error(self): | ||
| # """Test that unsupported providers raise ValueError.""" | ||
| # # Note: This would require modifying KaapiCompletionConfig to accept other providers | ||
| # # For now, this tests the error handling in the mapper | ||
| # # We'll create a mock config that bypasses validation | ||
| # from unittest.mock import MagicMock | ||
|
|
||
| # mock_config = MagicMock() | ||
| # mock_config.provider = "unsupported-provider" | ||
| # mock_config.params = KaapiLLMParams(model="some-model") | ||
|
|
||
| # with pytest.raises(ValueError) as exc_info: | ||
| # transform_kaapi_config_to_native(mock_config) | ||
|
|
||
| # assert "Unsupported provider" in str(exc_info.value) | ||
|
|
||
| # def test_transform_preserves_param_structure(self): | ||
| # """Test that transformation correctly structures nested parameters.""" | ||
| # kaapi_config = KaapiCompletionConfig( | ||
| # provider="openai", | ||
| # params=KaapiLLMParams( | ||
| # model="gpt-4", | ||
| # knowledge_base_ids=["vs_1", "vs_2", "vs_3"], | ||
| # max_num_results=15, | ||
| # ), | ||
| # ) | ||
|
|
||
| # result, warnings = transform_kaapi_config_to_native(kaapi_config) | ||
|
|
||
| # # Verify the nested structure is correct | ||
| # assert isinstance(result.params["tools"], list) | ||
| # assert isinstance(result.params["tools"][0], dict) | ||
| # assert isinstance(result.params["tools"][0]["vector_store_ids"], list) | ||
| # assert len(result.params["tools"][0]["vector_store_ids"]) == 3 | ||
| # assert warnings == [] |
There was a problem hiding this comment.
Entire test file is commented out - this needs to be addressed.
Commenting out an entire test suite (317 lines) without a tracking mechanism is problematic. These tests cover critical mapping logic from Kaapi-abstracted parameters to provider-native formats, including:
- Temperature/reasoning parameter handling for different model types
- Knowledge base ID mappings
- Config transformation logic
If the tests are failing due to interface changes from this PR, they should be updated rather than disabled. If they need temporary suppression, please:
- Add a TODO comment explaining why and linking to a tracking issue
- Consider using
pytest.mark.skip(reason="...")instead of commenting out code, which preserves test discovery and visibility
Would you like me to help identify what changes are needed to update these tests to work with the new type system ("type": "text" field, provider changes)?
🤖 Prompt for AI Agents
In `@backend/app/tests/services/llm/test_mappers.py` around lines 1 - 317, The
entire test suite in backend/app/tests/services/llm/test_mappers.py is commented
out; restore test visibility by either re-enabling the tests or explicitly
skipping them with a reason and tracking link. Undo the block comment (or
reintroduce the classes TestMapKaapiToOpenAIParams and
TestTransformKaapiConfigToNative) and run/fix failing assertions against
map_kaapi_to_openai_params and transform_kaapi_config_to_native if they break
due to the new type system, or if you must temporarily disable, add
pytest.mark.skip(reason="TODO: update tests for new type system, see
ISSUE-XXXXX") above each Test* class and add a TODO comment referencing the
issue; ensure the skip preserves the original test names and imports so future
fixes can target the exact failing assertions.
Comment on lines
+339
to
+373
| if latest_version: | ||
| # Extract the type and provider from the latest version | ||
| completion_config = latest_version.config_blob.get("completion", {}) | ||
| config_type = completion_config.get("type") | ||
| provider = completion_config.get("provider", "openai-native") | ||
|
|
||
| # Create a new config_blob maintaining the same type and provider | ||
| if provider in ["openai-native", "google-native"]: | ||
| config_blob = ConfigBlob( | ||
| completion=NativeCompletionConfig( | ||
| provider=provider, | ||
| type=config_type, | ||
| params={ | ||
| "model": completion_config.get("params", {}).get( | ||
| "model", "gpt-4" | ||
| ), | ||
| "temperature": 0.8, | ||
| "max_tokens": 1500, | ||
| }, | ||
| ) | ||
| ) | ||
| else: | ||
| # For Kaapi providers (openai, google) | ||
| config_blob = ConfigBlob( | ||
| completion=KaapiCompletionConfig( | ||
| provider=provider, | ||
| type=config_type, | ||
| params={ | ||
| "model": completion_config.get("params", {}).get( | ||
| "model", "gpt-4" | ||
| ), | ||
| "temperature": 0.8, | ||
| }, | ||
| ) | ||
| ) |
There was a problem hiding this comment.
Missing default for config_type could cause validation failure.
If latest_version.config_blob doesn't contain a type field (e.g., legacy data), config_type will be None, which would fail the Literal["text", "stt", "tts"] validation when passed to NativeCompletionConfig or KaapiCompletionConfig.
🔧 Suggested fix
if latest_version:
# Extract the type and provider from the latest version
completion_config = latest_version.config_blob.get("completion", {})
- config_type = completion_config.get("type")
+ config_type = completion_config.get("type", "text") # Default to "text" for legacy data
provider = completion_config.get("provider", "openai-native")📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| if latest_version: | |
| # Extract the type and provider from the latest version | |
| completion_config = latest_version.config_blob.get("completion", {}) | |
| config_type = completion_config.get("type") | |
| provider = completion_config.get("provider", "openai-native") | |
| # Create a new config_blob maintaining the same type and provider | |
| if provider in ["openai-native", "google-native"]: | |
| config_blob = ConfigBlob( | |
| completion=NativeCompletionConfig( | |
| provider=provider, | |
| type=config_type, | |
| params={ | |
| "model": completion_config.get("params", {}).get( | |
| "model", "gpt-4" | |
| ), | |
| "temperature": 0.8, | |
| "max_tokens": 1500, | |
| }, | |
| ) | |
| ) | |
| else: | |
| # For Kaapi providers (openai, google) | |
| config_blob = ConfigBlob( | |
| completion=KaapiCompletionConfig( | |
| provider=provider, | |
| type=config_type, | |
| params={ | |
| "model": completion_config.get("params", {}).get( | |
| "model", "gpt-4" | |
| ), | |
| "temperature": 0.8, | |
| }, | |
| ) | |
| ) | |
| if latest_version: | |
| # Extract the type and provider from the latest version | |
| completion_config = latest_version.config_blob.get("completion", {}) | |
| config_type = completion_config.get("type", "text") # Default to "text" for legacy data | |
| provider = completion_config.get("provider", "openai-native") | |
| # Create a new config_blob maintaining the same type and provider | |
| if provider in ["openai-native", "google-native"]: | |
| config_blob = ConfigBlob( | |
| completion=NativeCompletionConfig( | |
| provider=provider, | |
| type=config_type, | |
| params={ | |
| "model": completion_config.get("params", {}).get( | |
| "model", "gpt-4" | |
| ), | |
| "temperature": 0.8, | |
| "max_tokens": 1500, | |
| }, | |
| ) | |
| ) | |
| else: | |
| # For Kaapi providers (openai, google) | |
| config_blob = ConfigBlob( | |
| completion=KaapiCompletionConfig( | |
| provider=provider, | |
| type=config_type, | |
| params={ | |
| "model": completion_config.get("params", {}).get( | |
| "model", "gpt-4" | |
| ), | |
| "temperature": 0.8, | |
| }, | |
| ) | |
| ) |
🤖 Prompt for AI Agents
In `@backend/app/tests/utils/test_data.py` around lines 339 - 373, latest_version
may lack a "type" in its config_blob so config_type can be None and later fail
Literal validation; set a sensible default (e.g., "text") when extracting it
from completion_config and use that default when constructing ConfigBlob
instances for both NativeCompletionConfig and KaapiCompletionConfig (update the
assignment of config_type taken from completion_config.get("type") to fallback
to "text" and ensure NativeCompletionConfig and KaapiCompletionConfig are always
passed a non-None type).
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@backend/app/models/llm/request.py`:
- Around line 395-402: The LlmCall.provider Literal includes "anthropic" but
there is no anthropic provider implementation in the provider registry (only
"openai" and "google"), so either remove "anthropic" from the Literal on the
provider Field in the LlmCall model or prevent invalid values from being
persisted: update the provider Field definition in LlmCall.provider to remove
"anthropic" (so Literal["openai","google"]) OR add an explicit Pydantic
validator on LlmCall.provider that checks the value against the actual provider
registry (the same registry lookup used at runtime) and raises a ValidationError
if "anthropic" (or any unsupported provider) is supplied; if you plan to
implement anthropic, instead implement and register the anthropic provider in
the provider registry so lookups succeed.
In `@backend/app/tests/seed_data/seed_data.py`:
- Around line 21-22: Remove the unused imports Config and ConfigVersion from the
top of this module to clean up imports; locate the import list in seed_data.py
where Config and ConfigVersion are imported alongside other models and delete
those two symbols (or defer importing them only when you add
create_config/create_config_version seed functions), ensuring functions like
clear_database and existing seed_... helpers remain unchanged.
🧹 Nitpick comments (1)
backend/app/services/llm/providers/oai.py (1)
7-7: Consider grouping typing imports with standard library imports.The
typingimport should be placed before third-party imports (likeopenai) per PEP 8 import ordering conventions.+from typing import Any + import logging import openai from openai import OpenAI from openai.types.responses.response import Response -from typing import Any from app.models.llm import (
Comment on lines
+395
to
+402
| provider: Literal["openai", "google", "anthropic"] = Field( | ||
| ..., | ||
| sa_column=sa.Column( | ||
| sa.String, | ||
| nullable=False, | ||
| comment="AI provider: openai, google, anthropic", | ||
| ), | ||
| ) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# First, examine the exact code at the specified lines in the file
echo "=== Code at lines 395-402 in backend/app/models/llm/request.py ==="
sed -n '390,405p' backend/app/models/llm/request.py
echo -e "\n=== Code at line 155 in backend/app/models/llm/request.py (NativeCompletionConfig) ==="
sed -n '150,160p' backend/app/models/llm/request.py
echo -e "\n=== Code at line 175 in backend/app/models/llm/request.py (KaapiCompletionConfig) ==="
sed -n '170,180p' backend/app/models/llm/request.py
echo -e "\n=== Search for 'anthropic' in backend/app ==="
rg -n "anthropic" backend/app --type py -C 2Repository: ProjectTech4DevAI/kaapi-backend
Length of output: 5561
🏁 Script executed:
#!/bin/bash
# Check for anthropic provider implementation
echo "=== Searching for anthropic provider implementation ==="
find backend/app/services/llm/providers -name "*.py" | xargs ls -la
echo -e "\n=== Check for claude/anthropic provider files ==="
rg -n "class.*Provider" backend/app/services/llm/providers --type py | grep -i "claude\|anthropic"
echo -e "\n=== Full list of provider implementations ==="
ls -la backend/app/services/llm/providers/
echo -e "\n=== Check LlmCall class definition for coding guideline compliance ==="
# Get context around LlmCall class definition and provider field
sed -n '370,410p' backend/app/models/llm/request.pyRepository: ProjectTech4DevAI/kaapi-backend
Length of output: 2219
🏁 Script executed:
#!/bin/bash
echo "=== Check provider registry.py for anthropic ==="
cat backend/app/services/llm/providers/registry.py
echo -e "\n=== Check for error handling with unknown providers ==="
rg -n "provider.*not.*found\|unknown.*provider\|anthropic" backend/app/services/llm --type py -B2 -A2 | head -40
echo -e "\n=== Check if anthropic is dynamically loaded or imported anywhere ==="
rg -n "import.*anthropic\|from.*anthropic" backend/app --type pyRepository: ProjectTech4DevAI/kaapi-backend
Length of output: 4607
Remove "anthropic" from LlmCall.provider Literal or implement the missing anthropic provider.
LlmCall.provider accepts "anthropic" but the system has no anthropic provider implementation. The provider registry only supports "openai" and "google" and will raise ValueError if "anthropic" is used at runtime. This creates a data consistency issue where records can be persisted to the database but become unusable. Either remove "anthropic" from the Literal or implement the corresponding provider. If anthropic support is planned for the future, add explicit validation to prevent storing "anthropic" provider values until support is available.
🤖 Prompt for AI Agents
In `@backend/app/models/llm/request.py` around lines 395 - 402, The
LlmCall.provider Literal includes "anthropic" but there is no anthropic provider
implementation in the provider registry (only "openai" and "google"), so either
remove "anthropic" from the Literal on the provider Field in the LlmCall model
or prevent invalid values from being persisted: update the provider Field
definition in LlmCall.provider to remove "anthropic" (so
Literal["openai","google"]) OR add an explicit Pydantic validator on
LlmCall.provider that checks the value against the actual provider registry (the
same registry lookup used at runtime) and raises a ValidationError if
"anthropic" (or any unsupported provider) is supplied; if you plan to implement
anthropic, instead implement and register the anthropic provider in the provider
registry so lookups succeed.
Comment on lines
+21
to
+22
| Config, | ||
| ConfigVersion, |
There was a problem hiding this comment.
Unused imports: Config and ConfigVersion are not referenced in this file.
Neither Config nor ConfigVersion is used anywhere in this module—no seed functions create these entities, and they don't appear in clear_database or elsewhere. If these are placeholders for upcoming seed logic, consider deferring the import until the corresponding create_config/create_config_version functions are added; otherwise, remove them to keep imports clean.
🤖 Prompt for AI Agents
In `@backend/app/tests/seed_data/seed_data.py` around lines 21 - 22, Remove the
unused imports Config and ConfigVersion from the top of this module to clean up
imports; locate the import list in seed_data.py where Config and ConfigVersion
are imported alongside other models and delete those two symbols (or defer
importing them only when you add create_config/create_config_version seed
functions), ensuring functions like clear_database and existing seed_... helpers
remain unchanged.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Prajna1999
changed the title
Prajna1999
commented
Jan 28, 2026
| completion_type = completion_config.type | ||
| provider = completion_config.provider | ||
|
|
||
| generation_params = completion_config.params |
Collaborator
Author
There was a problem hiding this comment.
#TODO move this into a helper function.
Prajna1999
commented
Jan 28, 2026
| - NativeCompletionConfig with provider-native parameters ready for API | ||
| - List of warnings for suppressed/ignored parameters | ||
| """ | ||
| if kaapi_config.provider == "openai": |
Collaborator
Author
There was a problem hiding this comment.
#TODO convert this enums from string to proper enum types.
#TODO force per provider per type model, have a dictionary of sorts. Config YAML, dict etc. research
Prajna1999
commented
Jan 28, 2026
| job_id=job_id, | ||
| project_id=project_id, | ||
| organization_id=organization_id, | ||
| resolved_config=resolved_config_blob, |
Collaborator
Author
There was a problem hiding this comment.
raw config blob to negate conversion from openai, google to their native versions
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@backend/app/services/llm/providers/gai.py`:
- Around line 116-121: The Usage construction is dereferencing
response.usage_metadata fields directly which can raise AttributeError; update
the code that builds the Usage(...) (the Usage class instantiation using
response.usage_metadata.prompt_token_count, candidates_token_count,
total_token_count, thoughts_token_count) to first check if
response.usage_metadata is not None and then read each field (falling back to 0
or None as your convention) before passing values into Usage; i.e., compute safe
local values from response.usage_metadata with null checks and use those
variables in the Usage(...) call so missing usage_metadata cannot cause an
exception.
- Line 97: The uploaded gemini_file created via
self.client.files.upload(file=parsed_input) is never removed; update the logic
in the surrounding method (where gemini_file is used) to delete the remote file
with self.client.files.delete(name=gemini_file.name) after it's no longer
needed—preferably in a finally block to guarantee cleanup even on errors, and
handle/log any exceptions from self.client.files.delete so failures to clean up
don't crash the flow.
🧹 Nitpick comments (3)
backend/app/services/llm/providers/gai.py (3)
2-2: Unused importos.The
osmodule is imported but never used in this file.🧹 Suggested fix
import logging -import os from google import genai
99-101: Redundant condition check.The
if merged_instruction:check on line 100 is always true sincemerged_instructionis unconditionally assigned a non-empty string on either line 92 or 94.♻️ Simplified code
contents = [] - if merged_instruction: - contents.append(merged_instruction) + contents.append(merged_instruction) contents.append(gemini_file)
133-139: Remove unusedqueryparameter or document its purpose.The
query: QueryParamsparameter is accepted but never used in this method's implementation. While other providers (e.g.,OpenAIProvider) usequery.conversation,GoogleAIProvider.executedoes not reference it. Either remove the parameter (updating the interface as needed) or add a comment explaining why it's retained as part of the interface contract.
| merged_instruction = f"{lang_instruction}." | ||
|
|
||
| # Upload file and generate content | ||
| gemini_file = self.client.files.upload(file=parsed_input) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's examine the gai.py file to understand the context
cat -n backend/app/services/llm/providers/gai.py | head -150Repository: ProjectTech4DevAI/kaapi-backend
Length of output: 6166
🌐 Web query:
Google GenAI Python SDK file.delete() API documentation
💡 Result:
Google GenAI Python SDK — client.files.delete() (aka file.delete())
Package / install: google-genai (pip install google-genai). (ai.google.dev)
Python method (SDK):
- Call:
client.files.delete(name=..., config=None) - Purpose: Deletes a remotely stored file.
- Params:
name(str, required): file resource name likefiles/abc-123config(DeleteFileConfig, optional)
- Returns:
DeleteFileResponse - Async:
await client.aio.files.delete(name=..., config=None)(googleapis.github.io)
Underlying REST API (Gemini API):
DELETE https://generativelanguage.googleapis.com/v1beta/{name=files/*}- Request body is empty. (ai.google.dev)
Example (Python):
from google import genai
client = genai.Client()
myfile = client.files.upload(file="poem.txt")
client.files.delete(name=myfile.name) # e.g., "files/abc-123"References
- SDK
genai.files.Files.delete/AsyncFiles.deletedocs. (googleapis.github.io) - Gemini API “Method: files.delete” (endpoint + example). (ai.google.dev)
- Google GenAI SDK install info (
google-genai). (ai.google.dev)
Citations:
- 1: https://ai.google.dev/downloads?utm_source=openai
- 2: https://googleapis.github.io/python-genai/genai.html?utm_source=openai
- 3: https://ai.google.dev/api/files?utm_source=openai
- 4: https://ai.google.dev/api/files?utm_source=openai
- 5: https://googleapis.github.io/python-genai/genai.html?utm_source=openai
- 6: https://ai.google.dev/api/files?utm_source=openai
- 7: https://ai.google.dev/downloads?utm_source=openai
🏁 Script executed:
# Check if files.delete is used anywhere else in the codebase
rg "files\.delete" --type py
# Check if there are any other file cleanup patterns
rg "\.delete\(" backend/app/services/llm/ --type py -B 2 -A 2
# Verify usage_metadata null checks in the codebase
rg "usage_metadata" --type py -B 1 -A 1Repository: ProjectTech4DevAI/kaapi-backend
Length of output: 1245
🏁 Script executed:
# Check if there are any exception handling patterns around files.upload in the codebase
rg "files\.upload" --type py -B 3 -A 5Repository: ProjectTech4DevAI/kaapi-backend
Length of output: 3310
Missing cleanup for uploaded file.
The uploaded gemini_file is never deleted after use. This could lead to accumulation of files in Google's storage, potentially incurring costs and hitting quota limits. The Google GenAI SDK supports file deletion via client.files.delete(name=...).
🧹 Suggested fix
# Upload file and generate content
gemini_file = self.client.files.upload(file=parsed_input)
+ try:
+ contents = []
+ if merged_instruction:
+ contents.append(merged_instruction)
+ contents.append(gemini_file)
- contents = []
- if merged_instruction:
- contents.append(merged_instruction)
- contents.append(gemini_file)
+ response: GenerateContentResponse = self.client.models.generate_content(
+ model=model, contents=contents
+ )
+ finally:
+ try:
+ self.client.files.delete(name=gemini_file.name)
+ except Exception as cleanup_error:
+ logger.warning(
+ f"[GoogleAIProvider._execute_stt] Failed to cleanup uploaded file: {cleanup_error}"
+ )
- response: GenerateContentResponse = self.client.models.generate_content(
- model=model, contents=contents
- )🤖 Prompt for AI Agents
In `@backend/app/services/llm/providers/gai.py` at line 97, The uploaded
gemini_file created via self.client.files.upload(file=parsed_input) is never
removed; update the logic in the surrounding method (where gemini_file is used)
to delete the remote file with self.client.files.delete(name=gemini_file.name)
after it's no longer needed—preferably in a finally block to guarantee cleanup
even on errors, and handle/log any exceptions from self.client.files.delete so
failures to clean up don't crash the flow.
Comment on lines
+116
to
+121
| usage=Usage( | ||
| input_tokens=response.usage_metadata.prompt_token_count, | ||
| output_tokens=response.usage_metadata.candidates_token_count, | ||
| total_tokens=response.usage_metadata.total_token_count, | ||
| reasoning_tokens=response.usage_metadata.thoughts_token_count, | ||
| ), |
There was a problem hiding this comment.
Add null checks for usage_metadata fields.
Accessing response.usage_metadata.prompt_token_count and related fields without null checks could cause AttributeError if the response doesn't include usage metadata (e.g., on certain error conditions or API changes).
🛡️ Suggested defensive approach
+ usage_metadata = response.usage_metadata
llm_response = LLMCallResponse(
response=LLMResponse(
provider_response_id=response.response_id,
model=response.model_version,
provider=provider,
output=LLMOutput(text=response.text),
),
usage=Usage(
- input_tokens=response.usage_metadata.prompt_token_count,
- output_tokens=response.usage_metadata.candidates_token_count,
- total_tokens=response.usage_metadata.total_token_count,
- reasoning_tokens=response.usage_metadata.thoughts_token_count,
+ input_tokens=usage_metadata.prompt_token_count if usage_metadata else 0,
+ output_tokens=usage_metadata.candidates_token_count if usage_metadata else 0,
+ total_tokens=usage_metadata.total_token_count if usage_metadata else 0,
+ reasoning_tokens=usage_metadata.thoughts_token_count if usage_metadata else None,
),
)🤖 Prompt for AI Agents
In `@backend/app/services/llm/providers/gai.py` around lines 116 - 121, The Usage
construction is dereferencing response.usage_metadata fields directly which can
raise AttributeError; update the code that builds the Usage(...) (the Usage
class instantiation using response.usage_metadata.prompt_token_count,
candidates_token_count, total_token_count, thoughts_token_count) to first check
if response.usage_metadata is not None and then read each field (falling back to
0 or None as your convention) before passing values into Usage; i.e., compute
safe local values from response.usage_metadata with null checks and use those
variables in the Usage(...) call so missing usage_metadata cannot cause an
exception.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@backend/app/services/llm/providers/registry.py`:
- Around line 92-94: Update the except block that calls logger.error (the one
logging "Failed to initialize {provider_type} client") to prefix the message
with the current function name in square brackets; import inspect if needed and
change the log call to something like
logger.error(f"[{inspect.currentframe().f_code.co_name}] Failed to initialize
{provider_type} client: {e}", exc_info=True) so the function name appears before
the message, keeping the existing exc_info and the subsequent raise RuntimeError
unchanged.
🧹 Nitpick comments (2)
backend/pyproject.toml (1)
39-39:google-genai>=1.59.0is compatible with Python 3.12 and can be pinned with an upper bound for stability.Version 1.59.0 exists and is compatible with Python 3.12 (requires Python >=3.10). Consider adding an upper bound to avoid unplanned breaking changes in future releases.
♻️ Optional upper-bound pin
- "google-genai>=1.59.0", + "google-genai>=1.59.0,<2.0.0",backend/app/services/llm/jobs.py (1)
223-227: Consider usingmodel_copy()for forward-compatibility when reconstructingConfigBlob.While
ConfigBlobcurrently contains only thecompletionfield, usingmodel_copy(update={"completion": completion_config})would be more maintainable if additional fields are added in the future (the comments indicateclassifierandpre_filterare planned). The current approach is functionally correct, but the suggested defensive pattern would ensure any future fields are preserved during the reconstruction.
…ssistant_id handling
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Fix all issues with AI agents
In `@backend/app/alembic/versions/042_add_llm_call_table.py`:
- Around line 1-6: The migration docstring at the top of the file is out of sync
with the module constants: update the docstring header lines that currently read
"Revision ID: 041" and "Revises: 040" so they match the declared module-level
constants revision = "042" and down_revision = "041" (i.e., change the docstring
to Revision ID: 042 and Revises: 041) so the header and the revision constants
(revision / down_revision) are consistent.
- Around line 20-160: The function definition for upgrade is missing a return
type; update the def upgrade() signature to include an explicit return type (->
None) so it reads def upgrade() -> None: and ensure the function body remains
unchanged; apply the same pattern for any Alembic migration functions (e.g.,
downgrade) in this file if present to satisfy typing guidelines.
In `@backend/app/services/llm/providers/gai.py`:
- Around line 90-95: The file fails Black formatting around the string assembly
for forced_trascription_text and merged_instruction; run Black (or pre-commit)
to reformat the file so spacing/line breaks match Black style, and ensure the
variables forced_trascription_text, merged_instruction, instructions, and
lang_instruction remain unchanged semantically while fixing formatting
inconsistencies (e.g., string concatenation/spacing and line length) in the
function that builds merged_instruction.
- Around line 23-31: Update the constructor signature for __init__ in
backend/app/services/llm/providers/gai.py to include an explicit return type
annotation -> None (i.e., change "def __init__(self, client: genai.Client):" to
"def __init__(self, client: genai.Client) -> None:") and keep the body unchanged
(retain super().__init__(client) and self.client = client) so the initializer
conforms to the project's type-hinting guidelines.
Comment on lines
+1
to
+6
| """add_llm_call_table | ||
|
|
||
| Revision ID: 041 | ||
| Revises: 040 | ||
| Create Date: 2026-01-26 15:20:23.873332 | ||
|
|
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# Read the complete file to see header and revision constants
cat -n backend/app/alembic/versions/042_add_llm_call_table.py | head -30Repository: ProjectTech4DevAI/kaapi-backend
Length of output: 928
🏁 Script executed:
# Check for other migration files to understand the pattern
ls -1 backend/app/alembic/versions/*.py | sort | tail -5Repository: ProjectTech4DevAI/kaapi-backend
Length of output: 381
🏁 Script executed:
# Examine a previous migration file to see correct format
cat -n backend/app/alembic/versions/041_*.py | head -20Repository: ProjectTech4DevAI/kaapi-backend
Length of output: 569
Align migration header with the revision constants.
The docstring header (lines 3-4) shows Revision ID 041 / Revises 040, but the actual revision constants (lines 14-15) declare revision=042 / down_revision=041. Update the header to match:
✏️ Suggested fix
-Revision ID: 041
-Revises: 040
+Revision ID: 042
+Revises: 041🤖 Prompt for AI Agents
In `@backend/app/alembic/versions/042_add_llm_call_table.py` around lines 1 - 6,
The migration docstring at the top of the file is out of sync with the module
constants: update the docstring header lines that currently read "Revision ID:
041" and "Revises: 040" so they match the declared module-level constants
revision = "042" and down_revision = "041" (i.e., change the docstring to
Revision ID: 042 and Revises: 041) so the header and the revision constants
(revision / down_revision) are consistent.
Comment on lines
+20
to
+160
| def upgrade(): | ||
| # ### commands auto generated by Alembic - please adjust! ### | ||
| op.create_table( | ||
| "llm_call", | ||
| sa.Column( | ||
| "id", | ||
| sa.Uuid(), | ||
| nullable=False, | ||
| comment="Unique identifier for the LLM call record", | ||
| ), | ||
| sa.Column( | ||
| "job_id", | ||
| sa.Uuid(), | ||
| nullable=False, | ||
| comment="Reference to the parent job (status tracked in job table)", | ||
| ), | ||
| sa.Column( | ||
| "project_id", | ||
| sa.Integer(), | ||
| nullable=False, | ||
| comment="Reference to the project this LLM call belongs to", | ||
| ), | ||
| sa.Column( | ||
| "organization_id", | ||
| sa.Integer(), | ||
| nullable=False, | ||
| comment="Reference to the organization this LLM call belongs to", | ||
| ), | ||
| sa.Column( | ||
| "input", | ||
| sqlmodel.sql.sqltypes.AutoString(), | ||
| nullable=False, | ||
| comment="User input - text string, binary data, or file path for multimodal", | ||
| ), | ||
| sa.Column( | ||
| "input_type", | ||
| sa.String(), | ||
| nullable=False, | ||
| comment="Input type: text, audio, image", | ||
| ), | ||
| sa.Column( | ||
| "output_type", | ||
| sa.String(), | ||
| nullable=True, | ||
| comment="Expected output type: text, audio, image", | ||
| ), | ||
| sa.Column( | ||
| "provider", | ||
| sa.String(), | ||
| nullable=False, | ||
| comment="AI provider: openai, google, anthropic", | ||
| ), | ||
| sa.Column( | ||
| "model", | ||
| sqlmodel.sql.sqltypes.AutoString(), | ||
| nullable=False, | ||
| comment="Specific model used e.g. 'gpt-4o', 'gemini-2.5-pro'", | ||
| ), | ||
| sa.Column( | ||
| "provider_response_id", | ||
| sqlmodel.sql.sqltypes.AutoString(), | ||
| nullable=True, | ||
| comment="Original response ID from the provider (e.g., OpenAI's response ID)", | ||
| ), | ||
| sa.Column( | ||
| "content", | ||
| postgresql.JSONB(astext_type=sa.Text()), | ||
| nullable=True, | ||
| comment="Response content: {text: '...'}, {audio_bytes: '...'}, or {image: '...'}", | ||
| ), | ||
| sa.Column( | ||
| "usage", | ||
| postgresql.JSONB(astext_type=sa.Text()), | ||
| nullable=True, | ||
| comment="Token usage: {input_tokens, output_tokens, reasoning_tokens}", | ||
| ), | ||
| sa.Column( | ||
| "conversation_id", | ||
| sqlmodel.sql.sqltypes.AutoString(), | ||
| nullable=True, | ||
| comment="Identifier linking this response to its conversation thread", | ||
| ), | ||
| sa.Column( | ||
| "auto_create", | ||
| sa.Boolean(), | ||
| nullable=True, | ||
| comment="Whether to auto-create conversation if conversation_id doesn't exist (OpenAI specific)", | ||
| ), | ||
| sa.Column( | ||
| "config", | ||
| postgresql.JSONB(astext_type=sa.Text()), | ||
| nullable=True, | ||
| comment="Configuration: {config_id, config_version} for stored config OR {config_blob} for ad-hoc config", | ||
| ), | ||
| sa.Column( | ||
| "created_at", | ||
| sa.DateTime(), | ||
| nullable=False, | ||
| comment="Timestamp when the LLM call was created", | ||
| ), | ||
| sa.Column( | ||
| "updated_at", | ||
| sa.DateTime(), | ||
| nullable=False, | ||
| comment="Timestamp when the LLM call was last updated", | ||
| ), | ||
| sa.Column( | ||
| "deleted_at", | ||
| sa.DateTime(), | ||
| nullable=True, | ||
| comment="Timestamp when the record was soft-deleted", | ||
| ), | ||
| sa.ForeignKeyConstraint(["job_id"], ["job.id"], ondelete="CASCADE"), | ||
| sa.ForeignKeyConstraint( | ||
| ["organization_id"], ["organization.id"], ondelete="CASCADE" | ||
| ), | ||
| sa.ForeignKeyConstraint(["project_id"], ["project.id"], ondelete="CASCADE"), | ||
| sa.PrimaryKeyConstraint("id"), | ||
| ) | ||
| op.create_index( | ||
| "idx_llm_call_conversation_id", | ||
| "llm_call", | ||
| ["conversation_id"], | ||
| unique=False, | ||
| postgresql_where=sa.text("conversation_id IS NOT NULL AND deleted_at IS NULL"), | ||
| ) | ||
| op.create_index( | ||
| "idx_llm_call_job_id", | ||
| "llm_call", | ||
| ["job_id"], | ||
| unique=False, | ||
| postgresql_where=sa.text("deleted_at IS NULL"), | ||
| ) | ||
| op.alter_column( | ||
| "collection", | ||
| "llm_service_name", | ||
| existing_type=sa.VARCHAR(), | ||
| comment="Name of the LLM service", | ||
| existing_comment="Name of the LLM service provider", | ||
| existing_nullable=False, | ||
| ) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# Check the current state of the upgrade function in the migration file
head -30 backend/app/alembic/versions/042_add_llm_call_table.pyRepository: ProjectTech4DevAI/kaapi-backend
Length of output: 718
🏁 Script executed:
# Check other migration files to see if they have type hints
find backend/app/alembic/versions -name "*.py" -type f | head -5 | xargs -I {} sh -c 'echo "=== {} ===" && head -25 {}'Repository: ProjectTech4DevAI/kaapi-backend
Length of output: 3370
🏁 Script executed:
# Check if the file has a downgrade() function as well
grep -n "def downgrade" backend/app/alembic/versions/042_add_llm_call_table.pyRepository: ProjectTech4DevAI/kaapi-backend
Length of output: 95
🏁 Script executed:
# Check the downgrade function and surrounding context
sed -n '164,170p' backend/app/alembic/versions/042_add_llm_call_table.pyRepository: ProjectTech4DevAI/kaapi-backend
Length of output: 316
Add return type hint to upgrade function.
Line 20 defines upgrade without a return type; add -> None to meet the typing requirement.
✏️ Suggested fix
-def upgrade():
+def upgrade() -> None:Per coding guidelines, all **/*.py files must include type hints for function parameters and return values.
🤖 Prompt for AI Agents
In `@backend/app/alembic/versions/042_add_llm_call_table.py` around lines 20 -
160, The function definition for upgrade is missing a return type; update the
def upgrade() signature to include an explicit return type (-> None) so it reads
def upgrade() -> None: and ensure the function body remains unchanged; apply the
same pattern for any Alembic migration functions (e.g., downgrade) in this file
if present to satisfy typing guidelines.
Comment on lines
+23
to
+31
| def __init__(self, client: genai.Client): | ||
| """Initialize Google AI provider with client. | ||
|
|
||
| Args: | ||
| client: Google AI client instance | ||
| """ | ||
| super().__init__(client) | ||
| self.client = client | ||
|
|
There was a problem hiding this comment.
Add missing -> None return annotation on __init__.
The constructor lacks a return type annotation.
🔧 Proposed fix
- def __init__(self, client: genai.Client):
+ def __init__(self, client: genai.Client) -> None:As per coding guidelines: **/*.py: Always add type hints to all function parameters and return values in Python code.
🤖 Prompt for AI Agents
In `@backend/app/services/llm/providers/gai.py` around lines 23 - 31, Update the
constructor signature for __init__ in backend/app/services/llm/providers/gai.py
to include an explicit return type annotation -> None (i.e., change "def
__init__(self, client: genai.Client):" to "def __init__(self, client:
genai.Client) -> None:") and keep the body unchanged (retain
super().__init__(client) and self.client = client) so the initializer conforms
to the project's type-hinting guidelines.
Comment on lines
+90
to
+95
| forced_trascription_text="Only return transcribed text and no other text." | ||
| # Merge user instructions with language instructions | ||
| if instructions: | ||
| merged_instruction = f"{instructions}. {lang_instruction}. {forced_trascription_text}" | ||
| else: | ||
| merged_instruction = f"{lang_instruction}. {forced_trascription_text}" |
There was a problem hiding this comment.
Resolve Black formatting failure in this file.
CI reports Black reformatting; for example, Line 90 is not Black-compliant. Please run Black (or pre-commit) to fix formatting.
🧹 Example fix (Black will handle all formatting)
-forced_trascription_text="Only return transcribed text and no other text."
+forced_trascription_text = "Only return transcribed text and no other text."🤖 Prompt for AI Agents
In `@backend/app/services/llm/providers/gai.py` around lines 90 - 95, The file
fails Black formatting around the string assembly for forced_trascription_text
and merged_instruction; run Black (or pre-commit) to reformat the file so
spacing/line breaks match Black style, and ensure the variables
forced_trascription_text, merged_instruction, instructions, and lang_instruction
remain unchanged semantically while fixing formatting inconsistencies (e.g.,
string concatenation/spacing and line length) in the function that builds
merged_instruction.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Target issue is #515 and #556
Extend config management and Unified API endpoints for audio (STT and ASR) use cases.
Checklist
Before submitting a pull request, please ensure that you mark these task.
fastapi run --reload app/main.pyordocker compose upin the repository root and test.Notes
Revised configuration schema with config version could look something like this. Based on config_blob_completion_type: “text” | “tts” |”stt”, three different completion objects would need to be passed.
Modifications to schemas are highlighted.
llm_calltable storing essential metadata for llm_call request and response.autoflag in the STT endpoint.Test cases for mappers have been supressed because the usual behaviour created inconsistency for
provider.type=google/openaiandprovider.type=google-native/openai-native. Currently the provider execution functions take-nativeas the provider parameter regardless of whether stored config or ad-hoc has been passed in thellm/callendpoint. Ideally the key should follow after config provider not typecast to default-native. Since does not affect the STT feature, will take up after discussing the ideal behaviour in another PR.Also most file changes are auto created while fixing formatting. The real change files are few.
Summary by CodeRabbit
New Features
Chores
✏️ Tip: You can customize this high-level summary in your review settings.