From 35468af9d56d84e2e23d55e7f7dc842b99721ed7 Mon Sep 17 00:00:00 2001 From: Andrew Date: Thu, 17 Jul 2025 17:27:01 -0400 Subject: [PATCH 1/4] Create buildkite.mdx Create first version of buildkite.mdx for Turborepo CI Vendors documentation. --- .../docs/guides/ci-vendors/buildkite.mdx | 195 ++++++++++++++++++ 1 file changed, 195 insertions(+) create mode 100644 docs/site/content/docs/guides/ci-vendors/buildkite.mdx diff --git a/docs/site/content/docs/guides/ci-vendors/buildkite.mdx b/docs/site/content/docs/guides/ci-vendors/buildkite.mdx new file mode 100644 index 0000000000000..13381498d2ed1 --- /dev/null +++ b/docs/site/content/docs/guides/ci-vendors/buildkite.mdx @@ -0,0 +1,195 @@ +--- +title: Buildkite +description: Learn how to use Buildkite with Turborepo. +--- + +import { PackageManagerTabs, Tab } from '#components/tabs'; + +The following example shows how to use Turborepo with [Buildkite](https://buildkite.com/). + +For a given root `package.json`: + +```json title="./package.json" +{ + "name": "my-turborepo", + "scripts": { + "build": "turbo run build", + "test": "turbo run test" + }, + "devDependencies": { + "turbo": "latest" + } +} +``` + +And a `turbo.json`: + +```json title="./turbo.json" +{ + "$schema": "https://turborepo.com/schema.json", + "tasks": { + "build": { + "outputs": [".next/**", "!.next/cache/**"], + "dependsOn": ["^build"] + }, + "test": { + "dependsOn": ["^build"] + } + } +} +``` + +Create a file called `.buildkite/pipeline.yml` in your repository with the following contents: + + + + + + ```yaml title=".buildkite/pipeline.yml" + version: 2.1 + workflows: + test: + jobs: + - test + jobs: + test: + steps: + - checkout + - node/install-packages + - run: + command: npm i -g pnpm + environment: + TURBO_UI: "false" + - run: + command: pnpm build + environment: + TURBO_UI: "false" + - run: + command: pnpm test + environment: + TURBO_UI: "false" + ``` + + + + + ```yaml title=".buildkite/pipeline.yml" + version: 2.1 + workflows: + test: + jobs: + - test + jobs: + test: + steps: + - checkout + - node/install-packages: + pkg-manager: yarn + - run: + command: yarn build + environment: + TURBO_UI: "false" + - run: + command: yarn test + environment: + TURBO_UI: "false" + ``` + + + + + + ```yaml title=".buildkite/pipeline.yml" + version: 2.1 + workflows: + test: + jobs: + - test + jobs: + test: + steps: + - checkout + - node/install-packages + - run: + command: npm run build + environment: + TURBO_UI: "false" + + - run: + command: npm run test + environment: + TURBO_UI: "false" + ``` + + + + + + ```yaml title=".buildkite/pipeline.yml" + version: 2.1 + workflows: + test: + jobs: + - test + jobs: + test: + steps: + - checkout + - node/install-packages + - run: + command: npm i -g bun + environment: + TURBO_UI: "false" + - run: + command: bun run build + environment: + TURBO_UI: "false" + - run: + command: bun run test + environment: + TURBO_UI: "false" + ``` + + + + +## Create a Pipeline + +To create your pipeline in the Buildkite dashboard, you'll need to first upload the pipeline definition from your repository. + +1. Select **Pipelines** to navigate to the Buildkite dashboard. + +2. Select **New pipeline**. + +3. Enter your pipeline's details in the respective **Name** and **Description** fields. + +4. In the **Steps** editor, ensure there's a step to upload the definition from your repository: + +``` +steps: + - label: ":pipeline:" + command: buildkite-agent pipeline upload +``` + +5. Select **Create Pipeline**, then click **New Build**, then select **Create Build**. + +Run the pipeline whenever you make changes you want to verify. + +## Remote Caching + +To use Remote Caching, retrieve the team and token for the Remote Cache for your provider. In this example, we'll use [Vercel Remote Cache](https://vercel.com/docs/monorepos/remote-caching): + +- `TURBO_TOKEN` - The Bearer token to access the Remote Cache +- `TURBO_TEAM` - The account to which the monorepo belongs + +To use Vercel Remote Caching, you can get the value of these variables in a few steps: + +1. Create a Scoped Access Token to your account in the [Vercel Dashboard](https://vercel.com/account/tokens). Copy the value to a safe place. You'll need it in a moment. + +![Vercel Access Tokens](/images/docs/vercel-create-token.png) + +2. Make a second secret called `TURBO_TEAM` and enter [your Team URL](https://vercel.com/d?to=%2F%5Bteam%5D%2F%7E%2Fsettings&title=Find+Team+URL). Copy this value as well. Both values will be used in the next step. + +3. Select one of the following storage services plugins and follow the instructions to set both environment variables: +[AWS SSM](https://github.com/buildkite-plugins/aws-assume-role-with-web-identity-buildkite-plugin), [GC Secrets](https://github.com/buildkite-plugins/gcp-workload-identity-federation-buildkite-plugin), or [Hashicorp Vault](https://github.com/buildkite-plugins/vault-secrets-buildkite-plugin). +If you cannot use such a service, use the Buildkite agent's `environment` hook to export secrets to a job. For more information, read [Managing pipeline secrets](https://buildkite.com/docs/pipelines/security/secrets/managing). \ No newline at end of file From dc1d04bb14fdd769b1fb9634fa35b34cf989c821 Mon Sep 17 00:00:00 2001 From: Andrew Date: Thu, 17 Jul 2025 18:07:14 -0400 Subject: [PATCH 2/4] Add Buildkite page to Index page Add to index and fix linter error by naming code block. --- docs/site/content/docs/guides/ci-vendors/buildkite.mdx | 10 +++++----- docs/site/content/docs/guides/ci-vendors/index.mdx | 2 ++ 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/site/content/docs/guides/ci-vendors/buildkite.mdx b/docs/site/content/docs/guides/ci-vendors/buildkite.mdx index 13381498d2ed1..2757621e445ab 100644 --- a/docs/site/content/docs/guides/ci-vendors/buildkite.mdx +++ b/docs/site/content/docs/guides/ci-vendors/buildkite.mdx @@ -165,9 +165,9 @@ To create your pipeline in the Buildkite dashboard, you'll need to first upload 4. In the **Steps** editor, ensure there's a step to upload the definition from your repository: -``` +```yaml title=".buildkite/pipeline.yml" steps: - - label: ":pipeline:" + - label: ':pipeline:' command: buildkite-agent pipeline upload ``` @@ -190,6 +190,6 @@ To use Vercel Remote Caching, you can get the value of these variables in a few 2. Make a second secret called `TURBO_TEAM` and enter [your Team URL](https://vercel.com/d?to=%2F%5Bteam%5D%2F%7E%2Fsettings&title=Find+Team+URL). Copy this value as well. Both values will be used in the next step. -3. Select one of the following storage services plugins and follow the instructions to set both environment variables: -[AWS SSM](https://github.com/buildkite-plugins/aws-assume-role-with-web-identity-buildkite-plugin), [GC Secrets](https://github.com/buildkite-plugins/gcp-workload-identity-federation-buildkite-plugin), or [Hashicorp Vault](https://github.com/buildkite-plugins/vault-secrets-buildkite-plugin). -If you cannot use such a service, use the Buildkite agent's `environment` hook to export secrets to a job. For more information, read [Managing pipeline secrets](https://buildkite.com/docs/pipelines/security/secrets/managing). \ No newline at end of file +3. Select one of the following storage services plugins and follow the instructions to set both environment variables: + [AWS SSM](https://github.com/buildkite-plugins/aws-assume-role-with-web-identity-buildkite-plugin), [GC Secrets](https://github.com/buildkite-plugins/gcp-workload-identity-federation-buildkite-plugin), or [Hashicorp Vault](https://github.com/buildkite-plugins/vault-secrets-buildkite-plugin). + If you cannot use such a service, use the Buildkite agent's `environment` hook to export secrets to a job. For more information, read [Managing pipeline secrets](https://buildkite.com/docs/pipelines/security/secrets/managing). diff --git a/docs/site/content/docs/guides/ci-vendors/index.mdx b/docs/site/content/docs/guides/ci-vendors/index.mdx index e7b21d145ac47..1a908e9d8d92d 100644 --- a/docs/site/content/docs/guides/ci-vendors/index.mdx +++ b/docs/site/content/docs/guides/ci-vendors/index.mdx @@ -22,6 +22,8 @@ Turborepo not only speeds up builds, but also the rest of your Continuous Integr + + ## General Setup From bcb50919fe46a8f40196305e99c9289f53677cf8 Mon Sep 17 00:00:00 2001 From: Andrew Date: Thu, 17 Jul 2025 18:13:44 -0400 Subject: [PATCH 3/4] Fix copy on buildkite.mdx Clarify that the buildkite plugins are secrets storage services, not just storage services. --- docs/site/content/docs/guides/ci-vendors/buildkite.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/site/content/docs/guides/ci-vendors/buildkite.mdx b/docs/site/content/docs/guides/ci-vendors/buildkite.mdx index 2757621e445ab..29ff34ff498db 100644 --- a/docs/site/content/docs/guides/ci-vendors/buildkite.mdx +++ b/docs/site/content/docs/guides/ci-vendors/buildkite.mdx @@ -190,6 +190,6 @@ To use Vercel Remote Caching, you can get the value of these variables in a few 2. Make a second secret called `TURBO_TEAM` and enter [your Team URL](https://vercel.com/d?to=%2F%5Bteam%5D%2F%7E%2Fsettings&title=Find+Team+URL). Copy this value as well. Both values will be used in the next step. -3. Select one of the following storage services plugins and follow the instructions to set both environment variables: +3. Select one of the following secrets storage services plugins and follow the instructions to set both environment variables: [AWS SSM](https://github.com/buildkite-plugins/aws-assume-role-with-web-identity-buildkite-plugin), [GC Secrets](https://github.com/buildkite-plugins/gcp-workload-identity-federation-buildkite-plugin), or [Hashicorp Vault](https://github.com/buildkite-plugins/vault-secrets-buildkite-plugin). If you cannot use such a service, use the Buildkite agent's `environment` hook to export secrets to a job. For more information, read [Managing pipeline secrets](https://buildkite.com/docs/pipelines/security/secrets/managing). From c527baa4e51b16df6afad9216d9c7230de6e872f Mon Sep 17 00:00:00 2001 From: Christian Nunciato Date: Tue, 29 Jul 2025 18:31:11 -0700 Subject: [PATCH 4/4] Suggested edits --- .../docs/guides/ci-vendors/buildkite.mdx | 161 ++++++++---------- docs/site/dictionary.txt | 1 + 2 files changed, 73 insertions(+), 89 deletions(-) diff --git a/docs/site/content/docs/guides/ci-vendors/buildkite.mdx b/docs/site/content/docs/guides/ci-vendors/buildkite.mdx index 29ff34ff498db..95709c44bbb04 100644 --- a/docs/site/content/docs/guides/ci-vendors/buildkite.mdx +++ b/docs/site/content/docs/guides/ci-vendors/buildkite.mdx @@ -46,53 +46,32 @@ Create a file called `.buildkite/pipeline.yml` in your repository with the follo ```yaml title=".buildkite/pipeline.yml" - version: 2.1 - workflows: - test: - jobs: - - test - jobs: - test: - steps: - - checkout - - node/install-packages - - run: - command: npm i -g pnpm - environment: - TURBO_UI: "false" - - run: - command: pnpm build - environment: - TURBO_UI: "false" - - run: - command: pnpm test - environment: - TURBO_UI: "false" + steps: + - label: ":test_tube: Test" + command: | + pnpm install + pnpm test + + - label: ":hammer: Build" + command: | + pnpm install + pnpm build ``` ```yaml title=".buildkite/pipeline.yml" - version: 2.1 - workflows: - test: - jobs: - - test - jobs: - test: - steps: - - checkout - - node/install-packages: - pkg-manager: yarn - - run: - command: yarn build - environment: - TURBO_UI: "false" - - run: - command: yarn test - environment: - TURBO_UI: "false" + steps: + - label: ":test_tube: Test" + command: | + yarn + yarn test + + - label: ":hammer: Build" + command: | + yarn + yarn build ``` @@ -100,25 +79,16 @@ Create a file called `.buildkite/pipeline.yml` in your repository with the follo ```yaml title=".buildkite/pipeline.yml" - version: 2.1 - workflows: - test: - jobs: - - test - jobs: - test: - steps: - - checkout - - node/install-packages - - run: - command: npm run build - environment: - TURBO_UI: "false" - - - run: - command: npm run test - environment: - TURBO_UI: "false" + steps: + - label: ":test_tube: Test" + command: | + npm install + npm test + + - label: ":hammer: Build" + command: | + npm install + npm run build ``` @@ -126,28 +96,16 @@ Create a file called `.buildkite/pipeline.yml` in your repository with the follo ```yaml title=".buildkite/pipeline.yml" - version: 2.1 - workflows: - test: - jobs: - - test - jobs: - test: - steps: - - checkout - - node/install-packages - - run: - command: npm i -g bun - environment: - TURBO_UI: "false" - - run: - command: bun run build - environment: - TURBO_UI: "false" - - run: - command: bun run test - environment: - TURBO_UI: "false" + steps: + - label: ":test_tube: Test" + command: | + bun install + bun run test + + - label: ":hammer: Build" + command: | + bun install + bun run build ``` @@ -186,10 +144,35 @@ To use Vercel Remote Caching, you can get the value of these variables in a few 1. Create a Scoped Access Token to your account in the [Vercel Dashboard](https://vercel.com/account/tokens). Copy the value to a safe place. You'll need it in a moment. -![Vercel Access Tokens](/images/docs/vercel-create-token.png) - -2. Make a second secret called `TURBO_TEAM` and enter [your Team URL](https://vercel.com/d?to=%2F%5Bteam%5D%2F%7E%2Fsettings&title=Find+Team+URL). Copy this value as well. Both values will be used in the next step. - -3. Select one of the following secrets storage services plugins and follow the instructions to set both environment variables: - [AWS SSM](https://github.com/buildkite-plugins/aws-assume-role-with-web-identity-buildkite-plugin), [GC Secrets](https://github.com/buildkite-plugins/gcp-workload-identity-federation-buildkite-plugin), or [Hashicorp Vault](https://github.com/buildkite-plugins/vault-secrets-buildkite-plugin). - If you cannot use such a service, use the Buildkite agent's `environment` hook to export secrets to a job. For more information, read [Managing pipeline secrets](https://buildkite.com/docs/pipelines/security/secrets/managing). + ![Vercel Access Tokens](/images/docs/vercel-create-token.png) + +2. Obtain [your Team URL](https://vercel.com/d?to=%2F%5Bteam%5D%2F%7E%2Fsettings&title=Find+Team+URL) and copy its value as well. Both values will be used in the next step. + +3. In the Buildkite dashboard, create two new [Buildkite secrets](https://buildkite.com/docs/pipelines/security/secrets/buildkite-secrets), one for each value. Name them `TURBO_TOKEN` and `TURBO_TEAM`. + +4. Update `pipeline.yml` to fetch and apply `TURBO_TOKEN` and `TURBO_TEAM` as environment variables with the [Buildkite Secrets](https://github.com/buildkite-plugins/secrets-buildkite-plugin) plugin as shown. (For additional secret-management options, read [Managing pipeline secrets](https://buildkite.com/docs/pipelines/security/secrets/managing) in the Buildkite documentation.) + + ```yaml title=".buildkite/pipeline.yml" + steps: + - label: ':test_tube: Test' + command: | + npm install + npm test + plugins: + - secrets: + variables: + TURBO_TOKEN: TURBO_TOKEN + TURBO_TEAM: TURBO_TEAM + + - label: ':hammer: Build' + command: | + npm install + npm run build + plugins: + - secrets: + variables: + TURBO_TOKEN: TURBO_TOKEN + TURBO_TEAM: TURBO_TEAM + ``` + + Commit and push these changes to your repository, and on the next pipeline run, the secrets will be applied and Vercel Remote Caching will be active. diff --git a/docs/site/dictionary.txt b/docs/site/dictionary.txt index 1619a93d8f412..91baf92ed29f8 100644 --- a/docs/site/dictionary.txt +++ b/docs/site/dictionary.txt @@ -65,6 +65,7 @@ SHAs codemod GitLab CircleCI +Buildkite parallelize Parallelizing eslint-config-turbo