Just tried to build with this fix! Now it built successfull. But there are some new errors, during chatting through Anything. Seems like we need to define LLM model in the form near LocalAI address. Because now LocalAI try to search unexisted model:

Loading model in memory from file: /Users/andrey/sandbox/llm/current/models/model-placeholder

Anything screenshot

LocalAI logs

5:55PM DBG [whisper] Attempting to load
5:55PM DBG Loading model whisper from model-placeholder
5:55PM DBG Loading model in memory from file: /Users/andrey/sandbox/llm/current/models/model-placeholder
5:55PM DBG Loading GRPC Model whisper: {backendString:whisper model:model-placeholder threads:4 assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0x14000313a00 externalBackends:map[] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:false}
5:55PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/whisper
5:55PM DBG GRPC Service for model-placeholder will be running at: '127.0.0.1:51095'
5:55PM DBG GRPC Service state dir: /var/folders/f9/1b1jz83s4ysfn9zfncbsb8y40000gn/T/go-processmanager1786773264
5:55PM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:51095: connect: connection refused"
5:55PM DBG GRPC(model-placeholder-127.0.0.1:51095): stderr 2023/11/13 17:55:51 gRPC Server listening at 127.0.0.1:51095
5:55PM DBG GRPC Service Ready
5:55PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:model-placeholder ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:4 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/Users/andrey/sandbox/llm/current/models/model-placeholder Device: UseTriton:false ModelBaseName: UseFastTokenizer:false PipelineType: SchedulerType: CUDA:false CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0 Tokenizer: LoraBase: LoraAdapter: NoMulMatQ:false DraftModel: AudioPath: Quantization:}
5:55PM DBG [whisper] Fails: could not load model: rpc error: code = Unknown desc = stat /Users/andrey/sandbox/llm/current/models/model-placeholder: no such file or directory
5:55PM DBG [stablediffusion] Attempting to load
5:55PM DBG Loading model stablediffusion from model-placeholder
5:55PM DBG Loading model in memory from file: /Users/andrey/sandbox/llm/current/models/model-placeholder
5:55PM DBG Loading GRPC Model stablediffusion: {backendString:stablediffusion model:model-placeholder threads:4 assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0x14000313a00 externalBackends:map[] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:false}
5:55PM DBG [stablediffusion] Fails: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/stablediffusion. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
5:55PM DBG [piper] Attempting to load
5:55PM DBG Loading model piper from model-placeholder
5:55PM DBG Loading model in memory from file: /Users/andrey/sandbox/llm/current/models/model-placeholder
5:55PM DBG Loading GRPC Model piper: {backendString:piper model:model-placeholder threads:4 assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0x14000313a00 externalBackends:map[] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:false}
5:55PM DBG [piper] Fails: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/piper. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
[127.0.0.1]:50930 500 - POST /v1/chat/completions

Uh oh!

LocalAI for Embeddings #360

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions