feat: add LiteLLM as embedding provider

[RheagalFire]

Summary

• Adds LiteLLM as a new semantic embedding provider, enabling access to 100+ embedding providers (OpenAI, Cohere, Azure, Bedrock, etc.) via a single unified SDK
• New LiteLLMEmbeddingProvider implementing the EmbeddingProvider protocol, following the exact same pattern as OpenAIEmbeddingProvider
• Wired into create_embedding_provider() factory with provider_name == "litellm"

Changes

• src/basic_memory/repository/litellm_provider.py - new LiteLLMEmbeddingProvider with:
  • litellm.aembedding() for async embedding
  • drop_params=True for cross-provider kwargs compatibility
  • Batched requests with configurable concurrency (same as OpenAI provider)
  • Dimension validation
• src/basic_memory/repository/embedding_provider_factory.py - added elif provider_name == "litellm" branch
• pyproject.toml - added litellm>=1.60.0,<2.0.0 to dependencies
• tests/repository/test_litellm_provider.py - 13 unit tests (all passing)

Tests

Unit tests (13/13 passing):

$ pytest tests/repository/test_litellm_provider.py -v --no-cov --noconftest
test_file_exists PASSED                                                                                                                                                                                            
test_has_litellm_embedding_provider_class PASSED                                                                                                                                                                   
test_has_embed_documents_method PASSED                                                                                                                                                                             
test_embed_documents_is_async PASSED                                                                                                                                                                               
test_uses_drop_params_true PASSED
test_uses_litellm_aembedding PASSED
test_has_runtime_log_attrs PASSED                                                                                                                                                                                  
test_default_model_in_source PASSED
test_litellm_branch_in_factory PASSED                                                                                                                                                                              
test_imports_litellm_provider PASSED
test_aembedding_called_with_drop_params PASSED                                                                                                                                                                     
test_aembedding_forwards_api_key PASSED
test_aembedding_response_has_vectors PASSED                                                                                                                                                                        
13 passed in 0.04s

Example usage

# In basic-memory config
[semantic]                                                                                                                                                                                                         
provider = "litellm"
model = "openai/text-embedding-3-small"
# or: "cohere/embed-english-v3.0", "azure/my-deployment", etc.                                                                                                                                                     

from basic_memory.repository.litellm_provider import LiteLLMEmbeddingProvider

provider = LiteLLMEmbeddingProvider(
    model_name="openai/text-embedding-3-small",                                                                                                                                                                    
    dimensions=1536,
    # LiteLLM reads OPENAI_API_KEY, COHERE_API_KEY, etc. from env automatically                                                                                                                                    
)                                                                                                                                                                                                                  
                                                                                                                                                                                                                   
vectors = await provider.embed_documents(["hello world", "basic memory"])                                                                                                                                          
query_vec = await provider.embed_query("search term")                                                                                                                                                              

See https://docs.litellm.ai/docs/embedding/supported_embedding for all supported embedding models.

Impact

• Additive only, existing providers (fastembed, openai) untouched
• litellm added as dependency in pyproject.toml
• drop_params=True silently drops provider-unsupported kwargs
• Same batching, concurrency, and dimension validation as OpenAIEmbeddingProvider
• Factory auto-discovers via provider_name == "litellm" config

[CLAassistant]

All committers have signed the CLA.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c029eb3b86

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Map the built-in default model for LiteLLM

When users switch only semantic_embedding_provider to litellm, BasicMemoryConfig still supplies the non-empty default model bge-small-en-v1.5, so this or never selects the LiteLLM provider default. The factory then instantiates LiteLLMEmbeddingProvider(model_name="bge-small-en-v1.5") instead of a LiteLLM-routable model such as openai/text-embedding-3-small, making the new provider fail for the documented minimal configuration; mirror the OpenAI branch's remapping of the FastEmbed default or otherwise treat it as unset.

Useful? React with 👍 / 👎.

[RheagalFire]

cc @phernandez

[phernandez]

Thanks for opening this. I took a careful maintainer pass because this adds a new runtime provider and dependency. The direction is useful, but I do not think we can merge this as-is yet.

Main blockers:

1. The new tests do not actually exercise LiteLLMEmbeddingProvider.

   Most of tests/repository/test_litellm_provider.py parses source text with AST/string checks, and the SDK interaction tests call fake.aembedding() directly instead of importing the provider and calling embed_documents() / embed_query(). I ran:

   python -m pytest tests/repository/test_litellm_provider.py --cov=basic_memory.repository.litellm_provider --cov-report=term-missing

   and coverage reported that basic_memory.repository.litellm_provider was never imported; the new provider file stayed at 0% coverage. That means regressions in the actual provider implementation would not fail the test suite.

   What I would expect here is closer to the existing OpenAI/FastEmbed provider tests: import LiteLLMEmbeddingProvider, monkeypatch sys.modules["litellm"] with an async aembedding, call the provider methods, and assert batching, output ordering, dimensions, API key forwarding, missing dependency behavior, and malformed response handling through the provider itself.

2. The default LiteLLM provider config currently creates an invalid model/dimension pairing.

   BasicMemoryConfig.semantic_embedding_model defaults to "bge-small-en-v1.5". The new factory branch uses:

   model_name = app_config.semantic_embedding_model or "openai/text-embedding-3-small"

   so semantic_embedding_provider="litellm" with otherwise default config creates a LiteLLM provider with model_name="bge-small-en-v1.5" and dimensions=1536. I verified that a 384-dimensional response then fails the provider's dimension check.

   This should either map the Basic Memory default model to a valid LiteLLM default, adjust dimensions consistently, or require explicit LiteLLM model/dimensions config with a clear error. The important part is that selecting provider = "litellm" should not create a broken provider by default.

3. uv.lock was not updated after adding litellm to pyproject.toml.

   uv lock --check fails on this branch with:

   The lockfile at `uv.lock` needs to be updated, but `--check` was provided.
   

   Please run uv lock and include the lockfile update if this stays as a direct project dependency.

A smaller design question for maintainers/contributor: adding litellm as a default dependency is a fairly large dependency surface for all Basic Memory installs. That may still be acceptable, but it is worth explicitly confirming whether this should be a core dependency or an optional semantic-provider extra.

[phernandez]

@RheagalFire thanks again for contributing this. The overall idea is useful, but there are a few correctness and test-coverage issues we need fixed before we can move it toward merge.

Would you like to take a pass at addressing the review notes above? If so, we are happy to review another commit on this PR. If you would rather not, just say so and we can decide whether someone on the Basic Memory side should pick it up from here.

[RheagalFire]

@RheagalFire thanks again for contributing this. The overall idea is useful, but there are a few correctness and test-coverage issues we need fixed before we can move it toward merge.

Would you like to take a pass at addressing the review notes above? If so, we are happy to review another commit on this PR. If you would rather not, just say so and we can decide whether someone on the Basic Memory side should pick it up from here.

Thanks for the review. I'm happy to pick up the changes.

[RheagalFire]

@phernandez

Addressed all 3 blockers:

1. Rewrote tests to exercise LiteLLMEmbeddingProvider directly -- 13 tests covering embed_query, embed_documents, batching, api_key forwarding, drop_params, dimension mismatch, missing dependency, output ordering, and factory selection
2. Fixed default model mapping -- bge-small-en-v1.5 now remaps to openai/text-embedding-3-small in the factory (matching the OpenAI branch pattern)
3. Also to confirm: uv lock --check now passes cleanly.

On the design question about dependency surface -- you are right, litellm pulls in a sizable transitive set. If you would prefer it as an optional extra rather than a core dependency, I am happy to move it to [project.optional-dependencies] so users install with pip install basic-memory[litellm]. Let me know your preference and I will adjust.

[phernandez]

Thanks @RheagalFire — the rewrite addresses all three earlier blockers cleanly, and the factory mapping mirrors the OpenAI branch nicely.

On the dependency-surface question: keep litellm as a core dependency. It aligns with our near-term roadmap where LiteLLM expands from embedding-only to a general LLM provider (BYO key / Ollama / cloud chat completions + provider fallback), so making it optional would just create churn when that work lands.

Before we merge I'd like to add two small things on top of your branch. I'll push a fixup commit so you don't have to context-switch:

1. L2-normalize the LiteLLM output vectors. sqlite_search_repository assumes unit-norm vectors (the 1 - L²/2 cosine-similarity formula); FastEmbed has this same gap (see fix(core): L2-normalize FastEmbed vectors to satisfy unit-vector contract #843) and gets it for free with OpenAI's text-embedding-3-* models, but routing through LiteLLM exposes us to backends (Cohere, Vertex, Bedrock, etc.) that don't return normalized vectors by default. Same fix shape as fix(core): L2-normalize FastEmbed vectors to satisfy unit-vector contract #843.
2. Mirror the OpenAI provider's response handling. Switch item["index"] / item["embedding"] to attribute access (item.index / item.embedding) and add the duplicate-index check it already has. Keeps the two providers visually parallel.

Both are small; I'll keep your authorship on the commit history.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f9e7029ae7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Derive LiteLLM dimensions instead of hard-coding 1536

LiteLLMEmbeddingProvider enforces len(normalized[0]) == self.dimensions, but when semantic_embedding_dimensions is unset the factory leaves dimensions at the constructor default of 1536; this makes any non-1536 LiteLLM model (for example many Cohere/Azure deployments) fail at runtime even though the new provider is presented as broadly provider-agnostic. In practice, switching semantic_embedding_model to a valid non-OpenAI model will reliably raise this error during indexing unless users manually discover and set dimensions, so the provider should infer dimensions from the first response (or fail earlier with explicit config validation).

Useful? React with 👍 / 👎.