Repositories list

• llm-d-kv-cache-manager

  Public
  Distributed KV cache coordinator

  aiincubating

  Go

  •

  Other

  •19•43•15•3•Updated Jul 19, 2025Jul 19, 2025

• llm-d-benchmark

  Public
  llm-d benchmark scripts and tooling

  incubating

  Shell

  •

  Apache License 2.0

  •13•20•14•3•Updated Jul 18, 2025Jul 18, 2025

• llm-d

  Public
  llm-d is a Kubernetes-native high-performance distributed LLM inference framework

  Makefile

  •

  Apache License 2.0

  •113•1.4k•25•19•Updated Jul 18, 2025Jul 18, 2025

• llm-d-inference-scheduler

  Public
  Inference scheduler for llm-d

  kubernetesainetworking+ 2inferencegateway-api

  Go

  •

  Apache License 2.0

  •47•67•47•8•Updated Jul 18, 2025Jul 18, 2025

• llm-d-routing-sidecar

  Public
  Incubating P/D sidecar for llm-d

  incubating

  Go

  •

  Apache License 2.0

  •14•9•5•6•Updated Jul 18, 2025Jul 18, 2025

• llm-d-inference-sim

  Public
  A light weight vLLM simulator, for mocking out replicas.

  incubating

  Go

  •16•30•14•1•Updated Jul 17, 2025Jul 17, 2025

• llm-d.github.io

  Public
  Website for llm-d: This repository builds the website seen at llm-d.ai

  JavaScript

  •16•10•5•2•Updated Jul 14, 2025Jul 14, 2025

• llm-d-deployer

  Public
  Helm charts for llm-d

  helmhelm-chartllm-d

  Shell

  •

  Apache License 2.0

  •48•51•32•19•Updated Jul 13, 2025Jul 13, 2025

• llm-d-model-service

  Public
  Simplified model deployment on llm-d

  incubating

  Go

  •

  Apache License 2.0

  •14•26•29•6•Updated Jul 2, 2025Jul 2, 2025

• .github

  Public
  0•0•0•0•Updated Jun 3, 2025Jun 3, 2025

• llm-d-pd-utils

  Public
  Makefile

  •4•5•1•0•Updated May 21, 2025May 21, 2025

• community

  Public
  0•0•0•0•Updated May 15, 2025May 15, 2025