vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

Intelligent Mixture-of-Models Router for Efficient LLM Inference

A high-throughput and memory-efficient inference and serving engine for LLMs

LLM Instance gateway implementation.

Envoy AI Gateway is an open source project for using Envoy Gateway to handle request traffic from application clients to Generative AI services.

An Mixture-of-Models (MoM) router that intelligently directs OpenAI API requests to the most suitable models from a defined pool based on Semantic Understanding of the request's intent.

AIInfra（AI 基础设施）指AI系统从底层芯片等硬件，到上层软件栈支持AI大模型训练和推理。

AISystem 主要是指AI系统，包括AI芯片、AI编译器、AI推理和训练框架等AI全栈底层技术

Manages Envoy Proxy as a standalone or Kubernetes-based application gateway

Cost-efficient and pluggable Infrastructure components for GenAI inference

Repository for the next iteration of composite service (e.g. Ingress) and load balancing APIs.

Container Network Interface - networking for Linux containers

Train your AI self, amplify you, bridge the world

Next-generation Cloud Native Gateway

📜Fork for tracking CNCF projects

Cloud-native high-performance edge/middle/service proxy

Go implementation of data-plane-api

Generates Kubernetes CRD API reference documentation

This repository hosts the Multi-Cluster Service APIs. Providers can import packages in this repo to ensure their multi-cluster service controller implementations will be compatible with MCS data pl…

Hi, UESTCers ! If you want to join us, please take a look at the Group Application. 🚀🚀🚀

GitHub Organization management via Peribolos

Manage any layer-7 traffic in Istio Service Mesh.

Kiali project, observability for the Istio service mesh

Tools shared by several Istio repositories

A fast and efficient cloud native application runtime

Configure external DNS servers (AWS Route53, Google CloudDNS and others) for Kubernetes Ingresses and Services