llm-d incubation
Incubating components of llm-d, a Kubernetes-native high-performance distributed LLM inference framework
Popular repositories Loading
-
-
workload-variant-autoscaler
workload-variant-autoscaler PublicVariant optimization autoscaler for distributed inference workloads
-
llm-d-modelservice
llm-d-modelservice Publichelm charts for deploying models with llm-d
-
Repositories
Showing 7 of 7 repositories
- workload-variant-autoscaler Public
Variant optimization autoscaler for distributed inference workloads
llm-d-incubation/workload-variant-autoscaler’s past year of commit activity - hermes Public
Hermes is a cluster configuration scanning and self-test generation tool for llm-d inference workloads
llm-d-incubation/hermes’s past year of commit activity - llm-d-fast-model-actuation Public
llm-d-incubation/llm-d-fast-model-actuation’s past year of commit activity - llm-d-ci Public
llm-d-incubation/llm-d-ci’s past year of commit activity - ig-wva Public
Workload Variant Autoscaler is a service to compute the cost-optimal provisioning of heterogeneous accelerators for inference workloads with varying request latency objectives
llm-d-incubation/ig-wva’s past year of commit activity
Most used topics
Loading…