General bug fixes and improvements.

Introducing Lucy, a trace-native debugging assistant built directly into vLLora. Lucy reads your threads and traces end-to-end and tells you what went wrong, why it happened, and what to do next - without you having to manually scroll through hundreds of spans.

Read more about Lucy in our blog post.

What Lucy can help with:

• Diagnose failing runs (root cause, impact, and concrete fixes)
• Spot tool/schema mismatches and invalid tool arguments
• Detect contradictory or ambiguous prompts that lead to degraded behavior
• Highlight slow spans and summarize latency/cost drivers
• Catch silent degradations like output truncation

This release introduces Distri Agent Support, enabling distributed agent execution directly within vLLora. Distri agents are automatically downloaded, registered, and managed with real-time status reporting and health monitoring.

Additional improvements include enhanced Lucy configuration management for agents, OTLP metrics port configuration, project slug support across services, and run overview totals for better visibility into trace statistics.

Features​

• Add distri as component (2c2f382)
• Add Lucy configuration management for agents (f89df0e)
• Add OTLP metrics port configuration (4d150c4)
• Add project_slug support to VlloraMcp and related services (84d5082)
• Add support for distri agents (f79a1a2)
• Add totals to run overview (f6b8fa5)
• Dont block main thread for distri download and startup (911c328)
• Enhance Distri client functionality and configuration management (57cbad1)
• Enhance SQLite connection handling in DbPool (c5694ad)
• Implement agent registration with detailed status reporting (d742b20)
• Implement Lucy project handling and middleware integration (2d1dd1d)
• Support metrics in telemetery (#241) (3212952)
• Update distri based on current version and latest release (c5e990c)

Bug Fixes​

• Add missing property to api_invoke span (92bbaba)
• Fix API key usage for lucy (6fd6a12)
• Fix download of distri-server (5d0eed6)
• Fix run span timing (66c7807)
• getSpanContent return data (53763fc)
• Increase default limit for overview pagination from 100 to 1000 (ef941f4)
• Integrate KeyStorageError into GatewayApiError and update key retrieval in chat completion executors (39c94a6)
• Update provider info handling in model metadata (90dcf62)
• Use correct project in events stream (db39467)
• Use project slug for credentials retrieve (bc6c8dc)
• Use server port args from cli when no command is defined (af6df7d)
• when clone request in lucy project (738b62d)

vLLora now includes a CLI tool that brings trace inspection and debugging capabilities directly to your terminal. The CLI enables fast iteration, automation workflows, and local reproduction of LLM traces without leaving your terminal.

The CLI provides commands to:

• Search and filter traces by status, time range, model, and operation type
• Get run overviews with span trees and LLM call summaries
• Inspect individual LLM call payloads and responses
• Monitor system health with aggregated statistics

Learn more in the vLLora CLI documentation.

This release also introduces Custom Providers and Models, allowing you to register your own API endpoints and model identifiers. Connect to self-hosted inference engines (like Ollama or LocalAI), private enterprise gateways, or any OpenAI-compatible service using a namespaced format (provider/model-id). Configure providers and models through Settings or the Chat Model Selector.

Learn more in the Custom Providers and Models documentation.

Features​

• Add operation names filter to get traces API (41e3523)
• Enhance LLM call handling and tool summary structure (8283e74)
• Handle MCP functionallity through CLI (dc9915a)
• Remove trace_id from GetLlmCallParams and update related handling (4a0de09)
• Support custom providers, models and endpoints (#224) (148f9aa)

Bug Fixes​

• Fix response mapping (c64a13f)

vLLora now includes an MCP Server that exposes trace and run inspection as tools for coding agents. Debug, fix, and monitor your AI agents directly from your terminal or IDE by connecting Claude Desktop, Cursor, or any MCP-capable client to vLLora's MCP endpoint.

The MCP server provides tools to:

• Search and filter traces by status, time range, model, and more
• Get run overviews with span trees and error breadcrumbs
• Inspect individual LLM call payloads and responses
• Monitor system health with aggregated statistics

Learn more in the MCP Server documentation.

Features​

• Enhance MCP tools for traces information (bdd770f)
• Support tool calls event in responses API (fbcc376)

Bug Fixes​

• Allow to use custom endpoint for openai (9e44d55)
• thread cost calculation should be extracted from api_invoke only (c72c256)

vLLora now supports Custom Endpoints, allowing you to connect your own API endpoints to any provider. Simply provide your endpoint URL and API key through the Provider Keys UI, and vLLora will route requests to your custom endpoint instead of the default provider endpoint.

This feature enables you to:

• Use custom API gateways and proxies
• Connect to self-hosted models
• Route through OpenAI-compatible endpoints

Learn more in the Custom Endpoints documentation.

Features​

• Add custom endpoint support to ProviderInfo (eb9f43c)

Added support for the responses API for vllora_llm (see responses API docs) and related updates.

Features​

• Integrate responses API (#215) (00ac1d2)

Fixed bugs improving UI stability, cost calculations, and API endpoint behavior. Key improvements include fixes for debug mode span display, visual diagram interactions, and pagination handling.

Bug Fixes​

• Paused spans not displayed after page refresh in debug mode (#210)
• Cannot collapse content in InputNode (Visual Diagram) (#212)
• Cost calculation should only sum from api_invoke spans (#213)
• Inconsistent spacing between ThreadList rows (#214)
• POST /threads endpoint ignores JSON body pagination parameters (#216)
• Infinite scroll fires multiple duplicate requests and uses wrong offset (#217)

Introducing Debug Mode, an interactive debugging feature that lets you pause LLM requests before they're sent to the model. With debug mode enabled, you can inspect the full request payload, edit messages, parameters, and tool schemas in real time, then continue execution with your modifications—all without changing your application code.

Debug mode is perfect for debugging agent prompts, verifying model selection, inspecting tool schemas, and tuning parameters on the fly. Simply enable the breakpoint toggle in the Traces view, and every outgoing LLM request will pause for inspection and editing.

With debug mode you can:

• Inspect the model, messages, parameters, and tool schemas
• Continue with the original request
• Modify the request and send your edited version instead

Learn more in the Debug Mode documentation.

Features​

• add attributes field to GatewaySpanStartEvent and update related serialization logic (cdea3e8)
• add BreakpointReceiverGuard to handle span errors on receiver drop (ff1af55)
• enhance BreakpointManager to store and manage events by thread_id (97c4726)
• update BreakpointManager to include optional thread_id in requests and responses (4117bc7)

Bug Fixes​

• add offset check to prevent out-of-bounds access in TraceService (38db17e)
• allow too many arguments warning for GatewaySpanStartEvent constructor (4c31441)
• Clear listeners when breakpoints receiver is droped (4fb248e)
• Fix usage store in model call span (df3091c)

Introducing vllora_llm crate, a standalone Rust library that provides a unified interface for interacting with multiple LLM providers through the vLLora AI Gateway. The crate enables seamless chat completions across OpenAI-compatible, Anthropic, Gemini, and Bedrock providers, with built-in streaming support and telemetry integration.

use vllora_llm::client::VlloraLLMClient;
use vllora_llm::types::gateway::{ChatCompletionRequest, ChatCompletionMessage};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let request = ChatCompletionRequest {
        model: "gpt-4.1-mini".to_string(),
        messages: vec![
            ChatCompletionMessage::new_text("user".to_string(), "Say hello!".to_string()),
        ],
        ..Default::default()
    };

    let client = VlloraLLMClient::new();
    let response = client.completions().create(request).await?;
    Ok(())
}

Other improvements in this release​

• Enhanced breakpoint management with GlobalBreakpointStateEvent integration
• Improved error handling for multiple provider scenarios
• Asynchronous improvements to intercept functionality

Features​

• feat: Update examples (@karolisg) (08fbebb)
• feat: add GlobalBreakpointStateEvent and integrate it into GatewayEvent handling (@karolisg) (48d8d5f)

Bug Fixes​

• fix: update readme path and handle errors in model execution for multiple providers (@karolisg) (d20de5a)
• fix: make set_intercept_all asynchronous and ensure continuation of requests when disabled (@karolisg) (23b5cc5)

General bug fixes and improvements.

Introducing Clone Request & Experiments, a powerful new feature that enables you to A/B test prompts, compare models, and iterate on LLM requests directly from your traces. Clone any finished trace into an isolated experiment where you can safely tweak parameters, switch models, or modify prompts without affecting the original request.

The Experiment feature provides two editing modes: a Visual Editor for intuitive prompt tweaking and a JSON Editor for precise parameter control. Edit system and user messages, switch models on the fly, adjust temperature and other parameters, and run experiments with side-by-side comparison of tokens, costs, and outputs—all without ever touching your original trace.

Perfect for prompt engineering, model comparison, parameter tuning, and iterative debugging. Learn more in the Clone and Experiment documentation.

Other improvements in this release:

• Enhanced breakpoint management with global intercept and resume capabilities
• Improved tracing with better parent span ID handling and status recording
• Stream implementation refactoring for better LLM provider support
• Better output recording and model finish reason handling

Features​

• Feat: Refine status recording in Actix middleware for improved tracing (@karolisg) (3957790)
• Feat: Add output recording and model finish reason handling in OpenAI model event (@karolisg) (adffb0c)
• Feat: Improve tracing by adding parent span ID handling in Actix middleware (@karolisg) (c23ed70)
• Feat: Enhance breakpoint management with event handling for resume actions (@karolisg) (8137465)
• Feat: Add functionality to list all pending breakpoints and their requests (@karolisg) (23f4d68)
• Feat: Implement continue_all endpoint for managing multiple breakpoints (@karolisg) (4c5a117)
• Feat: Add global breakpoint management to intercept all requests (@karolisg) (4917e0c)
• Feat: Integrate breakpoint management into chat completion execution (@karolisg) (8b57e4d)
• #186 Feat: Refactor stream implementation for llm providers (@karolisg)

Bug Fixes​

• fix: fmt (@duonganhthu43) (d883050)
• fix: ensure usage is cloned for cost event and recorded in tracing (@karolisg) (232351b)

• Enhanced error handling for port conflicts during startup.
• Stream processing improvements with better tracing instrumentation for streaming responses.

Features​

• #184 feat: Handle used port scenario on startup (@karolisg)

Code Refactoring​

• refactor: Update chat completion execution to use ResultStream for streaming responses (@karolisg) (586b5a3)
• refactor: Enhance tracing instrumentation in stream handling for execute_stream and completions client (@karolisg) (2160619)

• Added thought signature support for Gemini 3 Pro models.
• General bug fixes and improvements.

Features​

• feat: Add support of thought signature (@karolisg) (929d094)

Bug Fixes​

• fix: Remove unnecessary API version from LangDB URL construction (@karolisg) (220dc8c)
• fix: remove unnecessary API version from LangDB URL construction (@MrunmayS) (948703d)

Code Refactoring​

• refactor: Restructure completions client by moving functionality to new modules and enhancing stream handling (@karolisg) (74fa1e2)
• refactor: Rename 'extra' field to 'extra_content' across multiple models for consistency (@karolisg) (cdc3c6e)
• refactor: Move llm to separate crate (@karolisg) (7d79fe4)

• General bug fixes and improvements.

Features​

• feat: Enhance chat completion with thread title extraction and improve thread service query structure (@karolisg) (e42e8f2)

• General bug fixes and improvements.

Introducing MCP Support, enabling seamless integration with Model Context Protocol servers. Connect your AI models to external tools, APIs, databases, and services through HTTP, SSE, or WebSocket transports. vLLora automatically discovers MCP tools, executes tool calls on your behalf, and traces all interactions—making it easy to extend your models with dynamic capabilities.

Other improvements in this release

• Full MCP server support with HTTP, SSE, and WebSocket transports, enabling dynamic tool execution and external system integration
• Embedding model support for Bedrock, Gemini, and OpenAI with comprehensive tracing and cost tracking
• Enhanced routing with conditional strategies, fallbacks, and maximum depth limits for complex request flows
• Improved cost tracking with cached input token pricing and enhanced usage monitoring across all providers
• Response caching improvements for better performance and cost optimization
• Thread handling with service integration and middleware support for conversation management
• Multi-tenant OpenTelemetry tracing with tenant-aware span management
• Claude Sonnet 4.5 model support
• Virtual model versioning via model@version syntax for flexible model selection
• Variables support in chat completions for dynamic prompt templating
• Enhanced model metadata with service level, release date, license, and knowledge cutoff information
• Google Vertex AI model fetching and integration
• Improved span management with RunSpanBuffer for efficient trace processing

Features​

• feat: Add JSON configuration with 8MB limit and clean up unused imports in http.rs (@karolisg) (4689128)
• feat: Introduce RunSpanBuffer for efficient span management and update trace service to utilize it (@karolisg) (483148c)
• feat: Update provider credentials handling to include GatewayTenant in update_provider_key function (@karolisg) (60e4fee)
• feat: Introduce thread handling module with service and middleware integration (@karolisg) (a5a9ccb)
• feat: add option to control UI opening on startup and update configuration handling (@karolisg) (e5ecdf4)
• #175 Feat: Tracing MCP server (@karolisg)
• feat: Add Slack notification for new brew releases (@karolisg) (a1632b2)
• feat: add Slack notification job to GitHub Actions workflow for release updates (@karolisg) (cf7b148)
• feat: Support claude sonnet 4.5 (@karolisg) (931dd56)
• feat: add cost field to span model for enhanced tracking capabilities (@karolisg) (64f3a7b)
• feat: add message_id field to span model and track elapsed time for processing streams across models (@karolisg) (d3dda6d)
• feat: add model and inference model names to tracing fields in TracedEmbedding (@karolisg) (6c096ca)
• feat: add LLMStartEvent logging for Bedrock, Gemini, and OpenAI embedding models (@karolisg) (f24fea3)
• feat: capture spans for Bedrock, Gemini, and OpenAI embeding models (@karolisg) (7197ae6)
• feat: Replace custom error handling with specific ModelError variants (@karolisg) (7839211)
• feat: Support base64 encoding in embeddings (@karolisg) (def4f90)
• feat: Add methods for token pricing in ModelPrice enum (@karolisg) (631664a)
• feat: Enhance OpenAI embeddings support with Azure integration and improve error handling (@karolisg) (16b401d)
• feat: Add Bedrock embeddings support and enhance error handling (@karolisg) (4d7858b)
• feat: Introduce Gemini embeddings model and enhance provider error handling (@karolisg) (785371a)
• feat: Add is_private field to model metadata for enhanced privacy control (@karolisg) (217ee73)
• feat: Add new embedding models and enhance model handling (@karolisg) (9ffa27a)
• #129 feat: Fetch models from Google Vertex (@karolisg)
• feat: Add async method to retrieve top model metadata by ranking (@karolisg) (6f45d3d)
• feat: Add model metadata support to chat completion execution (@karolisg) (e4d1b8f)
• feat: Add support for roles in ClickhouseHttp URL construction (@karolisg) (dabdfdd)
• feat: Extend ChatCompletionMessage struct to include optional fields for tool calls, refusal, tool call ID, and cache control (@karolisg) (529d4ea)
• feat: Add build_response method to AnthropicModel for constructing MessagesResponseBody from stream data (@karolisg) (1c42ef5)
• feat: Enhance GenerateContentResponse structure to include model_version and response_id, (@karolisg) (3d6d572)
• feat: Implement build_response method to construct CreateChatCompletionResponse from stream data for tracing purpose (@karolisg) (e305a0b)
• feat: Extend ModelMetadata with new fields for service level, release date, license, and knowledge cutoff date (@karolisg) (797aff5)
• feat: Add serde alias for InterceptorType Guardrail to support legacy "guard" identifier (@karolisg) (9dc6c43)
• #116 feat: Implement conditional routing strategy (@karolisg)
• feat: Integrate cache control logic into message content handling in MessageMapper (@karolisg) (416cb1d)
• feat: Update langdb_clust to version 0.9.4 and enhance token usage tracking in cost calculations (@karolisg) (489a4a9)
• feat: Add benchmark_info field to ModelMetadata (@karolisg) (1b7904f)
• feat: Introduce CacheControl struct and integrate it into message mapping for content types (@karolisg) (cc715da)
• feat: Add support for cached input token pricing in cost calculations and update related structures (@karolisg) (f1d4077)
• feat: Return template directly if no variables are provided in render function (@karolisg) (6adaf12)
• feat: Enhance logging by recording request payloads in Gemini client (@karolisg) (86a9875)
• feat: Add API_CALLS_BY_IP constant for enhanced rate limiting functionality (@karolisg) (5e5958c)
• feat: Add optional user_email field to RequestUser struct (@karolisg) (982c240)
• feat: Implement maximum depth limit for request routing in RoutedExecutor (@karolisg) (357ffca)
• feat: Handle max retries in request (@karolisg) (31d9d41)
• feat: add custom event for model events (@karolisg) (7299233)
• feat: Support project traces channels (@karolisg) (ce0efef)
• feat: add run lifecycle events and fix model usage tracking (@karolisg) (31875af)
• feat: implement tenant-aware OpenTelemetry trace (@karolisg) (ee48ae3)
• #92 feat: Basic responses support (@karolisg)
• #86 feat: Support http streamable transport (@karolisg)
• feat: Add options struct for prompt caching (@karolisg) (61181bc)
• feat: add description and keywords fields to thread (@karolisg) (11411ec)
• feat: Add key generation for transport type (@karolisg) (3238632)
• feat: add version support for virtual model retrieval via model@version syntax (@karolisg) (b1f4045)
• #73 feat: Add variables field to chat completions (@karolisg)
• #72 feat: Enhanced support for MCP servers (@karolisg)
• feat: Support azure url parsing and usage in client (@karolisg) (cb4c665)
• feat: Store tools results in spans (@karolisg) (4f8deff)
• feat: Store openai partner moderations guard metadata (@karolisg) (0415257)
• feat:Support openai moderation guardrails (@karolisg) (0285528)
• feat: Return 446 error on guard rejection (@karolisg) (e0dd668)
• #54 feat: Support custom endpoint for openai client (@karolisg)
• #46 feat: Implement guardrails system (@VG)
• feat: Support multiple identifiers in cost control (@karolisg) (302a84c)
• feat: Support all anthropic properties (@karolisg) (59dec4b)
• feat: Add support of anthropic thinking (@karolisg) (ff9815a)
• feat: Add extra to request (@karolisg) (a9d3c32)
• #29 feat: Support search in memory mcp tool (@VG)
• #28 feat: Use time windows for metrics (@karolisg)
• feat: Refactor targets usage for percentage router (@karolisg) (0445101)
• #21 feat: Support langdb key (@karolisg)
• #20 feat: Integrate routed execution with fallbacks (@karolisg)
• feat: Add missing gemini parameters (@karolisg) (5f1d15e)
• #15 feat: Improve UI (@karolisg)
• feat: Add model name and provider name to embeddings API (@karolisg) (452cb91)
• feat: Print provider and model name in logs (@karolisg) (3403968)
• #4 feat: Implement tui (@VG)
• #3 feat: Build for ubuntu and docker images (@VG)
• feat: Support .env variables for config (@karolisg) (65561e9)
• feat: Use in memory storage (@karolisg) (75bf2a1)
• feat: implement mcp support (@VG) (a97bc68)
• feat: Add rate limiting (@karolisg) (7066d87)
• feat: Add cost control and limit checker (@karolisg) (1300266)
• feat: Use user in openai requests (@karolisg) (96213dc)
• feat: Add api_invoke spans (@karolisg) (329ace4)
• feat: Enable otel when clickhouse config provided (@karolisg) (edbd16c)
• feat: Add database span writter (@karolisg) (72bd326)
• feat: Add clickhouse dependency (@karolisg) (78fe8ca)

Bug Fixes​

• fix: Change From implementation to TryFrom for GenericGroupResponse and handle errors with GatewayError (@karolisg) (ff34d81)
• fix: Update tools definition to use Option for type field (@karolisg) (1b8d21b)
• fix: Update trace ID and span ID conversion in span_to_db_trace function (@karolisg) (ccc8881)
• fix: Add serde aliases for ListGroupQueryParams and GroupByKey fields (@karolisg) (09b35e7)
• fix: Fix filter params aliases (@karolisg) (db0219f)
• fix: Support postgres table for mcp configs (@karolisg) (a6235b7)
• fix: correct version formatting in check_version function for header and update check (@karolisg) (554b79d)
• fix: format version header in session check_version function (@karolisg) (1aa7c88)
• #178 fix: enhance error logging in Actix OTEL middleware (@karolisg)
• fix: Invalidate Rust build cache when UI content changes (@duonganhthu43) (1e8dc3f)
• fix: fmt (@duonganhthu43) (a2d3c3e)
• fix: Use tool.name instead of label in tools span (@karolisg) (a3f2e6f)
• fix: Send only part of headers to session track (@karolisg) (137da3d)
• fix: build ui part (@duonganhthu43) (c1ac76a)
• fix: enhance usage tracking by adding raw_usage field and implementing content comparison in Message struct (@karolisg) (309a772)
• fix: Improve error message for invalid ModelCapability (@karolisg) (25973bf)
• fix: Update token calculation in OpenAIModel to include reasoning tokens (@karolisg) (7f54726)
• #140 fix: Add support for custom headers in transports. Fixes #135 (@karolisg)
• fix: Update routing logic to always return true for ErrorRate metric when no metrics are available (@karolisg) (ead8fc0)
• fix: Update apply_guardrails call to use slice reference for message to ensure proper handling (@karolisg) (f00c778)
• fix: Improve error handling in stream_chunks by logging send errors for GatewayApiError (@karolisg) (74c6d54)
• fix: Update GatewayApiError handling for ModelError to return BAD_REQUEST for ModelNotFound (@karolisg) (bacb9ef)
• fix: Correct input token cost calculation by ensuring cached tokens are properly subtracted (@karolisg) (f42a00b)
• fix: Workaround xai tool calls issue (@karolisg) (da836f1)
• fix: Add workaround for XAI bug (@karolisg) (aeef458)
• fix: Handle template error during rendering (@karolisg) (45d960c)
• fix: Fix operation name for model spans (@karolisg) (ba68f69)
• fix: Fix retries handle in llm calls (@karolisg) (7c5354b)
• fix: Fix retries logic (@karolisg) (3edfae3)
• fix: Handle thought signature in gemini response (@karolisg) (f1e1501)
• fix: Fix duplicated tools labels in gemini tools spans (@karolisg) (233b100)
• fix: Properly handle model calls traces in gemini (@karolisg) (f57d522)
• fix: Empty required parameters list (@karolisg) (4e887b3)
• fix: Fix required default value (@karolisg) (c2ebf62)
• fix: Fix tracing for cached responses (@karolisg) (b9422cd)
• fix: handle cache response errors gracefully in gateway service (@karolisg) (38eba17)
• fix: Handle nullable types in gemini (@karolisg) (4a5a3ad)
• fix: Fix nested gemini structured output schema (@karolisg) (e97a592)
• fix: Fix gemini structured output generation (@karolisg) (45b376a)
• Fix: Fix gemini tool calls (@karolisg) (5331e2c)
• fix: Store call information in anthropic span when system prompt is missing (@karolisg) (c856d9a)
• fix: Handle empty arguments (@karolisg) (694a040)
• fix: Fix tags extraction (@karolisg) (732d872)
• fix: Add index to tool calls (@karolisg) (1963d20)
• fix: Support proxied engine types (@karolisg) (30d04b0)
• fix: Fix nested json schema (@karolisg) (2fcecc7)
• fix: Fix models name in GET /models API (@karolisg) (af93b7c)
• #38 fix: Fix ttft capturing (@karolisg)
• fix: Fix gemini call when message is empty (@karolisg) (77dfb2e)
• fix: Return formated error on bedrock validation (@karolisg) (b36434a)
• fix: Create secure context for script router (@karolisg) (5adfcd7)
• fix: Fix serialization of user properties (@karolisg) (af3e20b)
• fix: Fix routing direction for tps and requests metrics (@karolisg) (74317bb)
• fix: Return authorization error on invalid key (@karolisg) (08b870c)
• fix: Fix map tool names to labels in openai (@karolisg) (f434dd7)
• #26 fix: Fix langdb config load (@karolisg)
• fix: Add router span (@karolisg) (789261c)
• #22 fix: Fix response format usage (@karolisg)
• fix: Fix model name in models_call span (@karolisg) (ba57d2b)
• fix: Store inference model name in model call span (@karolisg) (a446bee)
• fix: Fix tags in tracing (@karolisg) (f0f6ffd)
• Fix: Fix provider name in tracing (@karolisg) (8754804)
• #18 fix: Fix provider name (@karolisg)
• fix: Improve error handling in loading config (@karolisg) (d4fbc25)
• fix: Fix tracing for openai and deepseek (@karolisg) (679aaa8)
• fix: Fix connection to mcp servers (@karolisg) (562852d)
• fix: Fix tonic shutdown on ctrl+c (@karolisg) (d83828f)

Code Refactoring​

• refactor: Remove mcp_server module and associated TavilySearch implementation (@karolisg) (ae7dbb1)
• refactor: Remove unnecessary logging in thread service middleware (@karolisg) (f775dc9)
• refactor: Migrate codebase to vllora functionality (@karolisg) (78575b3)
• refactor: Enhance embedding handling and introduce new model structure (@karolisg) (013b214)
• refactor: Remove unused EngineType variants from the engine module (@karolisg) (69e12a7)
• refactor: Clean up code formatting and remove unused dependency (@karolisg) (7b19a1b)
• refactor: Update model handling and enhance Azure OpenAI integration (@karolisg) (868ffef)
• refactor: Consolidate CredentialsIdent usage across modules and enhance cost calculation (@karolisg) (8cd96e8)
• refactor: Integrate price and credentials identification into model handling (@karolisg) (88fbae0)
• refactor: Update ModelIOFormats enum to include PartialEq derive and remove Bedrock model file (@karolisg) (65152c3)
• refactor: Introduce BedrockCredentials type and update AWS credential handling (@karolisg) (fa28214)
• refactor: Enhance metric routing to include default metrics for missing models (@karolisg) (54c7e2c)
• refactor: Update default context size in Bedrock model provider to zero (@karolisg) (1211217)
• refactor: Enhance Bedrock model ID formatting with region prefix (@karolisg) (8f80b92)
• refactor: Update Anthropic model to handle optional system messages (@karolisg) (b3859a8)
• refactor: Simplify message sending in stream_chunks function (@karolisg) (00b657e)
• refactor: Enhance Bedrock model ID handling with version replacement (@karolisg) (ec5ab40)
• refactor: Update Bedrock model provider to skip specific model ARNs and use model ARN for metadata (@karolisg) (d4034c1)
• refactor: Simplify Bedrock model ID handling and remove unused model ARN assignment (@karolisg) (968ba0a)
• refactor: Improve AWS region configuration handling in get_user_shared_config (@karolisg) (36c62c9)
• refactor: Add warning logs for Bedrock model name during conversation (@karolisg) (c4a7ed0)
• refactor: Remove debug logging for Bedrock client credentials (@karolisg) (5991121)
• refactor: Update BedrockModel to utilize ChatCompletionMessageWithFinishReason and adjust region configuration (@karolisg) (b7fb560)
• refactor: Remove unnecessary error logging in GeminiModel response handling (@karolisg) (343df18)
• refactor: Enhance OpenAIModel response handling with finish reason and usage tracking (@karolisg) (91aaf32)
• refactor: Update ChatCompletionMessage to include finish reason and adjust related model implementations (@karolisg) (f884c47)
• refactor: Simplify match expression for ModelError in GatewayApiError and streamline error logging in stream_chunks (@karolisg) (68deec6)
• refactor: Remove redundant logging of system messages in AnthropicModel (@karolisg) (f6d3a73)
• refactor: Simplify ProjectTraceMap type by removing receiver from tuple (@karolisg) (ea1c088)
• refactor: Add alias for InMemory transport type in McpTransportType enum (@karolisg) (04710a1)
• refactor: Clean up unused app_data references in ApiServer and adjust telemetry imports (@karolisg) (a40a683)
• refactor: Remove unused TraceMap references from RoutedExecutor and related modules (@karolisg) (2fe3067)
• refactor: integrate InMemoryMetricsRepository into routing logic (@karolisg) (329e153)
• refactor: split chat completion streaming into separate chunks for delta, finish reason and usage (@karolisg) (c67d478)
• refactor: Fix use of variables (@karolisg) (5e5ad6b)
• refactor: rename PromptCache to ResponseCache for better clarity and consistency (@karolisg) (bc9677f)
• refactor: move caching logic to dedicated cache module and update response types (@karolisg) (708797e)

• General bug fixes and improvements.

Features​

• #175 Feat: Tracing MCP server (@karolisg)

• General bug fixes and improvements.

• General bug fixes and improvements.

Bug Fixes​

• fix: Invalidate Rust build cache when UI content changes (@duonganhthu43) (82facb8)
• fix: fmt (@duonganhthu43) (8d5dd49)