这是indexloc提供的服务,不要输入任何密码
Skip to content

StreamOption.includeUsage = true causes OpenAiService.mapStreamToAccumulator to throw java.lang.IndexOutOfBoundsException #60

@MishaKopaniov

Description

@MishaKopaniov

When StreamOption.includeUsage = true, the last usage chunk contains no choices, resulting in an IndexOutOfBoundsException.

Example of the Last Usage Chunk:
{ "id": "chatcmpl-A4yfRYHaCuVyGtk5r0DnV9donhXja", "object": "chat.completion.chunk", "created": 1725749881, "model": "gpt-4o-2024-05-13", "choices": [], "usage": { "prompt_tokens": 611, "completion_tokens": 13, "total_tokens": 624 }, "system_fingerprint": "fp_992d1ea92d" }

The OpenAiService.mapStreamToAccumulator method attempts to access the first choice in the choices array with the following code:
ChatCompletionChoice firstChoice = chunk.getChoices().get(0);

However, this code assumes that there is always at least one choice available. When choices is empty, this leads to a java.lang.IndexOutOfBoundsException.

Although my understanding of the model is limited, it seems reasonable to either:

  1. Ignore the usage chunk during mapStreamToAccumulator.
  2. Add usage data to AssistantMessage model

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions