这是indexloc提供的服务,不要输入任何密码
Skip to content

LocalAI streaming text overflow #416

@lunamidori5

Description

@lunamidori5

There is a bug that crashes when Localai sends a chunk with streaming turned on

Cannonball results 347 -> 148 tokens.
Cannonball results 196 -> 147 tokens.
Cannonball results 309 -> 148 tokens.
/app/server/utils/chats/stream.js:256
          chunk += message;
                   ^

RangeError: Invalid string length
    at IncomingMessage.<anonymous> (/app/server/utils/chats/stream.js:256:20)
    at IncomingMessage.emit (node:events:517:28)
    at addChunk (node:internal/streams/readable:368:12)
    at readableAddChunk (node:internal/streams/readable:341:9)
    at Readable.push (node:internal/streams/readable:278:10)
    at HTTPParser.parserOnBody (node:_http_common:131:24)
    at TLSSocket.socketOnData (node:_http_client:541:22)
    at TLSSocket.emit (node:events:517:28)
    at addChunk (node:internal/streams/readable:368:12)
    at readableAddChunk (node:internal/streams/readable:341:9)
    ```
    
    I added a temp fix by just having it in a try if it fails, just set the chunk to "" 

Metadata

Metadata

Assignees

No one assigned

    Labels

    blockedpossible bugBug was reported but is not confirmed or is unable to be replicated.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions