Patch Ollama Streaming chunk issues #500
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Replace stream/sync chats with Langchain interface for now connect #499 ref: #495 (comment)
server/utils/AiProviders/ollama/index.jsIn the #convertToLangchainPrototypes(chats = []) {
const {
HumanMessage,
SystemMessage,
AIMessage,
} = require("langchain/schema");
const langchainChats = [];
const roleToMessageMap = {
'system': SystemMessage,
'user': HumanMessage,
'assistant': AIMessage
};
for (const chat of chats) {
const MessageClass = roleToMessageMap[chat.role];
if (MessageClass) {
langchainChats.push(new MessageClass({ content: chat.content }));
}
}
return langchainChats;
}server/utils/chats/stream.jsIn the try {
for await (const chunk of stream) {
const content = chunk.hasOwnProperty("content") ? chunk.content : chunk;
fullText += content;
writeResponseChunk(response, {
uuid,
sources: [],
type: "textResponseChunk",
textResponse: content,
close: false,
error: false,
});
}
} catch (error) {
console.error(`Error handling stream: ${error}`);
// Handle the error appropriately
} |
Comment on lines
+43
to
+57
| for (const chat of chats) { | ||
| switch (chat.role) { | ||
| case "system": | ||
| langchainChats.push(new SystemMessage({ content: chat.content })); | ||
| break; | ||
| case "user": | ||
| langchainChats.push(new HumanMessage({ content: chat.content })); | ||
| break; | ||
| case "assistant": | ||
| langchainChats.push(new AIMessage({ content: chat.content })); | ||
| break; | ||
| default: | ||
| break; | ||
| } | ||
| } |
There was a problem hiding this comment.
Replaced the switch statement in the #convertToLangchainPrototypes method with a map object for better readability and performance.
Suggested change
| for (const chat of chats) { | |
| switch (chat.role) { | |
| case "system": | |
| langchainChats.push(new SystemMessage({ content: chat.content })); | |
| break; | |
| case "user": | |
| langchainChats.push(new HumanMessage({ content: chat.content })); | |
| break; | |
| case "assistant": | |
| langchainChats.push(new AIMessage({ content: chat.content })); | |
| break; | |
| default: | |
| break; | |
| } | |
| } | |
| const roleToMessageMap = { | |
| 'system': SystemMessage, | |
| 'user': HumanMessage, | |
| 'assistant': AIMessage | |
| }; | |
| for (const chat of chats) { | |
| const MessageClass = roleToMessageMap[chat.role]; | |
| if (MessageClass) { | |
| langchainChats.push(new MessageClass({ content: chat.content })); | |
| } | |
| } |
|
Confirmed working! |
cabwds
pushed a commit
to cabwds/anything-llm
that referenced
this pull request
Jul 3, 2025
Replace stream/sync chats with Langchain interface for now connect Mintplex-Labs#499 ref: Mintplex-Labs#495 (comment)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Replace stream/sync chats with Langchain interface for now
connect #499
ref: #495 (comment)