Tags: signcl/litellm
Tags
[Fix] Allow using HTTP_ Proxy settings with trust_env (BerriAI#12066) * allow using trust_env * add docs on how to use HTTP_PROXY * docs AIOHTTP_TRUST_ENV * test_aiohttp_transport_trust_env_setting * docs fix
feat: implement Perplexity citation tokens and search queries cost ca… …lculation (BerriAI#11938) * feat: add citation_cost_per_token and search_queries_cost_per_1000 fields to ModelInfoBase - Add citation_cost_per_token field to ModelInfoBase for Perplexity citation token costs - Add search_queries_cost_per_1000 field to ModelInfoBase for Perplexity search query costs - Update _get_model_info_helper to include these fields in model info responses - Enables proper cost calculation for Perplexity-specific usage metrics * feat: update Perplexity sonar-deep-research model pricing configuration - Update input/output token costs to / per million tokens respectively - Add reasoning token cost at per million tokens - Add citation_cost_per_token at per million tokens (same as input) - Add search_queries_cost_per_1000 at /bin/zsh.005 per 1000 search queries - Remove deprecated search_context_cost_per_query structure - Aligns with Perplexity's updated pricing model for deep research capabilities * feat: implement Perplexity-specific cost calculator - Create cost_per_token function for Perplexity provider - Calculate standard input/output token costs - Add citation token cost calculation using citation_cost_per_token rate - Add reasoning token cost calculation with fallback to completion_tokens_details - Add search query cost calculation using search_queries_cost_per_1000 rate - Return separate prompt_cost and completion_cost for accurate billing - Handles all Perplexity-specific usage metrics: citation_tokens, num_search_queries, reasoning_tokens * feat: integrate Perplexity cost calculator with main cost calculation system - Import perplexity_cost_per_token function in main cost calculator - Add perplexity provider case to cost_per_token function - Enables automatic routing of Perplexity cost calculations to provider-specific logic - Maintains compatibility with existing cost calculation patterns - Supports all Perplexity-specific cost metrics through unified interface * feat: enhance Perplexity response transformation to extract cost-related fields - Override transform_response method to extract Perplexity-specific usage fields - Add _enhance_usage_with_perplexity_fields method to process API responses - Extract citation_tokens from citations array using character-based estimation (~4 chars/token) - Extract num_search_queries from both usage field and root level with priority handling - Create usage object when none exists to ensure cost fields are always captured - Handle empty citations and missing fields gracefully - Enables automatic extraction of cost metrics from Perplexity API responses * test: add comprehensive test suite for Perplexity cost calculation features Add 82 comprehensive tests across 3 test files: - test_perplexity_cost_calculator.py (59 tests): * Cost calculation with citation tokens, search queries, reasoning tokens * Various combinations and edge cases * Integration with main cost calculator * Model info access and validation * Zero values and missing fields handling - test_perplexity_chat_transformation.py (12 tests): * Citation token extraction from API responses * Search query extraction from usage and root fields * Priority handling and field aggregation * Empty citations and missing fields handling * Token estimation accuracy validation - test_perplexity_integration.py (11 tests): * End-to-end cost calculation workflows * High-volume and edge case scenarios * Model info integration validation * Case-insensitive provider matching * Transformation preservation of existing fields Ensures reliability and correctness of all Perplexity cost features with comprehensive coverage of happy path, edge cases, and error conditions. * fix: remove unused Union import from Perplexity transformation - Remove unused typing.Union import from litellm/llms/perplexity/chat/transformation.py - Fixes F401 linting error: 'typing.Union imported but unused' - Maintains only necessary imports: Any, List, Optional, Tuple * Fix JSON schema validation and use web_search_requests field - Add citation_cost_per_token and search_queries_cost_per_1000 to JSON schema - Update Perplexity transformation to use web_search_requests in PromptTokensDetailsWrapper - Update Perplexity cost calculator to read from web_search_requests field - Maintain backward compatibility while using standard LiteLLM fields * Fix type errors in Perplexity cost calculator - Add null checks for token counts and cost values to prevent None multiplication errors - Use .get() with fallback values instead of direct dictionary access - Ensure all arithmetic operations handle None values safely This fixes the failing job 44517525148 type errors. * Refactor Perplexity cost calculation tests to improve accuracy and consistency - Replace absolute difference assertions with math.isclose for better precision in cost comparisons - Update tests to utilize PromptTokensDetailsWrapper for handling web search requests - Ensure all test cases correctly reflect the new structure of usage fields, enhancing clarity and maintainability * fix: address type hinting issues in PerplexityChatConfig usage handling - Add type ignore comments to model_response.usage assignments to resolve type checking errors - Ensures compatibility with type definitions while maintaining existing functionality * Update model pricing configuration in JSON backup - Add citation_cost_per_token and search_queries_cost_per_1000 fields to enhance cost tracking - Remove deprecated search_context_cost_per_query structure to streamline pricing model - Aligns with recent updates in Perplexity's pricing strategy * Update search queries cost structure in model_prices_and_context_window.json to use search_context_cost_per_query * Refactor search queries cost structure in model_prices_and_context_window_backup.json and update related code to use search_queries_cost_per_query. Remove deprecated search_queries_cost_per_1000 references across model info and tests. * Enhance cost calculation in cost_calculator.py by introducing a safe float casting function to handle potential None and invalid values. Update cost calculations for input, citation, output, reasoning, and search query tokens to use this new function, ensuring more robust handling of model pricing data. * Refactor cost calculation in cost_calculator.py to support both legacy and current search cost keys. Enhance handling of search cost values by accommodating both dictionary and float formats, ensuring robust cost computation for search queries. * Update test cases to reflect changes in cost structure, renaming search_queries_cost_per_query to search_context_cost_per_query for consistency with recent refactor. Ensure assertions in tests align with updated cost keys. * Update test_perplexity_integration.py to rename search_queries_cost_per_query to search_context_cost_per_query, ensuring consistency with recent cost structure changes. Adjust assertions to align with updated cost keys.
docs(custom_llm_server.md): document anthropic custom llm translation
feat: add workload identity federation between GCP and AWS (BerriAI#1… …0210)
test(internal_user_endpoints.py): catch user already exists in team e… …rror
PreviousNext