diff --git a/pages/configuration.mdx b/pages/configuration.mdx index 4c80bf3..fd14009 100644 --- a/pages/configuration.mdx +++ b/pages/configuration.mdx @@ -66,7 +66,7 @@ This feature is most useful for when you have AnythingLLM as a simple sub-servic **NOTE:** You should enable these configurations _after_ you have enabled multi-user mode, created at least one `admin` user, and have completed the onboarding flow - in the AnythingLLM instance. + in the AnythingLLM instance.
Do **not** enabled these configurations before you have done this or else you may find yourself soft-locked out of the instance until you disable these flags.
@@ -195,3 +195,29 @@ COLLECTOR_ALLOW_ANY_IP="true" ### Disable Fully remove or comment out the `COLLECTOR_ALLOW_ANY_IP` environment variable to return to the default behavior. + +## Disable Streaming for Generic OpenAI Provider + + + **Note:** This setting only affects the Generic OpenAI provider and does not impact other LLM providers. Use this when your custom LLM endpoint does not support streaming responses. + + +Modification of the `GENERIC_OPENAI_STREAMING_DISABLED` environment variable allows you to disable streaming responses when using the Generic OpenAI provider. This is particularly useful when you're using a custom LLM that doesn't support streaming responses. + +By default, AnythingLLM attempts to use streaming for a better user experience. However, some custom LLM implementations may not support this feature, resulting in errors or unexpected behavior. + +When this setting is enabled, all responses from your Generic OpenAI provider will be returned as complete responses rather than streamed chunks. + +### Enable + +Set the `GENERIC_OPENAI_STREAMING_DISABLED` environment variable to **`"true"`** to enable. +_It must be set to a string value of `"true"` to be effective._ + +```bash +# Must be set to a string value of "true" to be effective. +GENERIC_OPENAI_STREAMING_DISABLED="true" +``` + +### Disable + +Fully remove or comment out the `GENERIC_OPENAI_STREAMING_DISABLED` environment variable to return to the default behavior of using streaming responses.