这是indexloc提供的服务,不要输入任何密码
Skip to content

[BUG]: Qdrant integration fails on document upload with more than 500 chunks #4544

@aeromech-1

Description

@aeromech-1

How are you running AnythingLLM?

All versions

What happened?

When uploading a large document that gets split into more than 500 chunks (the default Qdrant batch limit), the embedding process fails. The addDocumentToNamespace function in server/utils/vectorDbProviders/qdrant/index.js correctly splits the vectors into chunks but then incorrectly attempts to upsert all vectors at once, rather than iterating through the prepared chunks.

Are there known steps to reproduce?

To Reproduce

  1. Set up AnythingLLM with Qdrant as the vector database.
  2. Use an embedding model with a small chunk size (e.g., 1024).
  3. Upload a large document that will generate more than 500 vector chunks.
  4. Observe the server logs for an error during the client.upsert operation.

Expected behavior
The document should be successfully embedded by sending the vector chunks to Qdrant in multiple batches, each no larger than the batch limit.

Additional context
The issue is in the loop that processes toChunks(vectors, 500). The fix is to use the chunk variable inside the loop to create the batch for client.upsert, rather than using the full submission object.

Metadata

Metadata

Assignees

No one assigned

    Labels

    possible bugBug was reported but is not confirmed or is unable to be replicated.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions