AI SDK - GitHub Models

The GitHub Models provider for the AI SDK gives you access to the GitHub Models catalog of large language models, designed for building modern web applications. GitHub Models has support for multiple model families, like xAI, OpenAI and Meta.

Important

GitHub Models is currently in pre-release and subject to GitHub Pre-release License Terms.

To learn more about integrating GitHub Models into your projects, visit the GitHub Models documentation.

Setup

The GitHub Models provider is available in the @github/models NPM package. You can install it with:

npm i @github/models

Quick Start

To get started, you'll need a GitHub PAT (personal access token).

import {githubModels} from '@github/models'
import {generateText} from 'ai'

const result = await generateText({
  model: githubModels('meta/meta-llama-3.1-8b-instruct'),
  prompt: 'Write a haiku about programming.',
})

console.log(result.text)

Tip

GitHub Models includes free usage for all accounts. For higher usage limits, you can opt into paid usage.

If you need a customized setup, you can import createGithubModels from @github/models and create a provider instance with your settings:

import {createGithubModels} from '@github/models'

const githubModels = createGithubModels({
  org: 'my-organization',
})

Configuration Options

• apiKey string

  Your GitHub Personal Access Token or Fine-Grained Token with read:models scope. Defaults to the GITHUB_TOKEN environment variable.

• org string

  Organization to attribute API usage to (optional).

• baseURL string

  Custom API endpoint URL. Defaults to https://models.github.ai/inference.

• headers Record<string, string>

  Additional headers to include with requests.

• fetch (input: RequestInfo, init?: RequestInit) => Promise

  Custom fetch implementation. Defaults to the global fetch function. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of GitHub's trademarks or logos is subject to and must follow GitHub's Brand Toolkit. Use of GitHub's trademarks or logos in modified versions of this project must not cause confusion or imply GitHub sponsorship. Any use of third-party trademarks or logos are subject to those third-party’s policies.

License

Distributed under the MIT License. See LICENSE for more information.