You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When StreamOption.includeUsage = true, the last usage chunk contains no choices, resulting in an IndexOutOfBoundsException.
Example of the Last Usage Chunk: { "id": "chatcmpl-A4yfRYHaCuVyGtk5r0DnV9donhXja", "object": "chat.completion.chunk", "created": 1725749881, "model": "gpt-4o-2024-05-13", "choices": [], "usage": { "prompt_tokens": 611, "completion_tokens": 13, "total_tokens": 624 }, "system_fingerprint": "fp_992d1ea92d" }
The OpenAiService.mapStreamToAccumulator method attempts to access the first choice in the choices array with the following code: ChatCompletionChoice firstChoice = chunk.getChoices().get(0);
However, this code assumes that there is always at least one choice available. When choices is empty, this leads to a java.lang.IndexOutOfBoundsException.
Although my understanding of the model is limited, it seems reasonable to either:
Ignore the usage chunk during mapStreamToAccumulator.