这是indexloc提供的服务，不要输入任何密码
Skip to content

Disable generic OpenAI streaming via env #177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 1, 2025
Merged

Disable generic OpenAI streaming via env #177

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1f13042
add docs for disabling generic openai provider streaming
shatfield4 Jul 1, 2025
18a79ba
add docs for disabling generic openai provider streaming
shatfield4 Jul 1, 2025
b846ba3
fix typo in docs for GENERIC_OPENAI_STREAMING_DISABLED
shatfield4 Jul 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 27 additions & 1 deletion pages/configuration.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ This feature is most useful for when you have AnythingLLM as a simple sub-servic

<Callout type="warning" emoji="️⚠️">
**NOTE:** You should enable these configurations _after_ you have enabled multi-user mode, created at least one `admin` user, and have completed the onboarding flow
in the AnythingLLM instance.
in the AnythingLLM instance.
<br/>
Do **not** enabled these configurations before you have done this or else you may find yourself soft-locked out of the instance until you disable these flags.
</Callout>
Expand Down Expand Up @@ -195,3 +195,29 @@ COLLECTOR_ALLOW_ANY_IP="true"
### Disable

Fully remove or comment out the `COLLECTOR_ALLOW_ANY_IP` environment variable to return to the default behavior.

## Disable Streaming for Generic OpenAI Provider

<Callout type="info" emoji="️💡">
**Note:** This setting only affects the Generic OpenAI provider and does not impact other LLM providers. Use this when your custom LLM endpoint does not support streaming responses.
</Callout>

Modification of the `GENERIC_OPENAI_STREAMING_DISABLED` environment variable allows you to disable streaming responses when using the Generic OpenAI provider. This is particularly useful when you're using a custom LLM that doesn't support streaming responses.

By default, AnythingLLM attempts to use streaming for a better user experience. However, some custom LLM implementations may not support this feature, resulting in errors or unexpected behavior.

When this setting is enabled, all responses from your Generic OpenAI provider will be returned as complete responses rather than streamed chunks.

### Enable

Set the `GENERIC_OPENAI_STREAMING_DISABLED` environment variable to **`"true"`** to enable.
_It must be set to a string value of `"true"` to be effective._

```bash
# Must be set to a string value of "true" to be effective.
GENERIC_OPENAI_STREAMING_DISABLED="true"
```

### Disable

Fully remove or comment out the `GENERIC_OPENAI_STREAMING_DISABLED` environment variable to return to the default behavior of using streaming responses.