Skip to content

test: improve test coverage for agent implementations #23

@dkargatzis

Description

@dkargatzis

Overview

Increase test coverage for agent implementations to ensure reliability and maintainability. Current coverage is low for several agent components, particularly LangGraph nodes and complex execution paths.

Current Coverage Gaps

High Priority

  • src/agents/engine_agent/nodes.py - 13% coverage (LangGraph nodes, validation strategies, LLM evaluation)
  • src/agents/acknowledgment_agent/agent.py - 18% coverage (acknowledgment evaluation logic)
  • src/agents/acknowledgment_agent/prompts.py - 21% coverage
  • src/agents/feasibility_agent/nodes.py - 15% coverage (YAML generation, feasibility analysis)

Medium Priority

  • src/agents/engine_agent/prompts.py - 48% coverage
  • src/agents/base.py - 79% coverage (retry logic, timeout handling edge cases)
  • src/agents/factory.py - 47% coverage (error handling paths)

Requirements

Test Coverage Goals

  • Achieve minimum 80% coverage for all agent modules
  • Focus on critical execution paths and error handling
  • Test LangGraph node interactions and state transitions
  • Validate structured output parsing and error recovery

Test Areas

Engine Agent

  • Test all validation strategy selection paths (static, hybrid, LLM)
  • Test concurrent validator execution
  • Test LLM evaluation with various response formats
  • Test timeout and retry scenarios
  • Test rule description conversion and validation

Feasibility Agent

  • Test YAML generation for various rule types
  • Test feasibility analysis for edge cases
  • Test error handling in analysis and generation nodes
  • Test confidence score calculation

Acknowledgment Agent

  • Test acknowledgment evaluation logic
  • Test context analysis and decision making
  • Test various acknowledgment scenarios (valid, invalid, edge cases)
  • Test prompt generation and response parsing

Base Agent

  • Test retry logic with different failure patterns
  • Test timeout handling for various operation types
  • Test structured output retry scenarios
  • Test error propagation and metadata

Testing Standards

  • Use pytest.mark.asyncio for async tests
  • Mock external dependencies (LLM, GitHub API)
  • Test both success and failure paths
  • Include edge cases and boundary conditions
  • Follow existing test patterns in tests/unit/

Implementation Notes

  • Extend existing test files in tests/unit/
  • Use existing mocking patterns (AsyncMock, MagicMock)
  • Ensure tests run without network calls
  • Add fixtures for common test data
  • Document test scenarios and expected behaviors

Acceptance Criteria

  • All agent modules achieve minimum 80% test coverage
  • Critical execution paths have comprehensive test coverage
  • Error handling paths are tested
  • Tests follow existing patterns and conventions
  • All tests pass in CI without external dependencies

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions