The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

Inference Llama 2 in one file of pure C

Inference code for CodeLlama models

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

Fast and memory-efficient exact attention

BCQ tutorial for transformers

LLMs for your CLI

C++ implementation of ChatGLM-6B & ChatGLM2-6B & ChatGLM3 & GLM4(V)

A high-throughput and memory-efficient inference and serving engine for LLMs

Seamlessly integrate LLMs into scikit-learn.

The unofficial python package that returns response of Google Bard through cookie value.

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…

LLMPerf is a library for validating and benchmarking LLMs

Letta is the platform for building stateful agents: open AI with advanced memory that can learn and self-improve over time.

Block-sparse primitives for PyTorch

[ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling