这是indexloc提供的服务，不要输入任何密码
Skip to content

Cover gitignored env files in docs and examples #3270

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jan 18, 2023
Merged

Cover gitignored env files in docs and examples #3270

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
42947bf
Cover gitignored env files in docs and examples
ericmatthys Oct 21, 2022
d9dcf83
fixup! Cover gitignored env files in docs and examples
ericmatthys Nov 17, 2022
fc74eea
add section for dotenv
mehulkar Jan 11, 2023
0c7cbf2
Merge branch 'main' into gitignored-env-files
mehulkar Jan 11, 2023
afc4558
Minor copy update.
anthonyshew Jan 17, 2023
f36afc0
Merge branch 'main' into gitignored-env-files
anthonyshew Jan 17, 2023
6bba5cd
Merge branch 'main' into gitignored-env-files
kodiakhq[bot] Jan 18, 2023
2dbfa28
Merge branch 'main' into gitignored-env-files
kodiakhq[bot] Jan 18, 2023
b2431c1
Merge branch 'main' into gitignored-env-files
kodiakhq[bot] Jan 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 48 additions & 6 deletions docs/pages/repo/docs/core-concepts/caching.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Let's say you run a `build` task with Turborepo using `turbo run build`:

![](/images/docs/cache-miss.png)

1. Turborepo will **evaluate the inputs to your task** (by default all non-gitignored files in the workspace folder) and **turn them into a hash** (e.g. `78awdk123`).
1. Turborepo will **evaluate the inputs to your task** (by default all non-git-ignored files in the workspace folder) and **turn them into a hash** (e.g. `78awdk123`).

2. **Check the local filesystem cache** for a folder named with the hash (e.g.`./node_modules/.cache/turbo/78awdk123`).

Expand Down Expand Up @@ -352,20 +352,61 @@ The plugin will warn you if you are using non-framework-related environment vari

### Invisible Environment Variables

Since Turborepo runs _before_ your tasks, it is possible for your tasks to create or mutate environment variables after `turbo` has already calculated the hash for a particular task. For example, consider this `package.json`:
Since Turborepo runs _before_ your tasks, it is possible for your tasks to create or mutate environment
variables after `turbo` has already calculated the hash for a particular task. For example, consider this `package.json`:

```json
{
"scripts": {
"build": "NEXT_PUBLIC_GA_ID=UA-00000000-0 next build",
"test": "node -r dotenv/config test.js"
"build": "NEXT_PUBLIC_GA_ID=UA-00000000-0 next build"
}
}
```

`turbo`, having calculated a task hash prior to invoking the `build` script, will be unable to discover the `NEXT_PUBLIC_GA_ID=UA-00000000-0` environment variable and thus unable to partition the cache based upon that, or any environment variable configured by `dotenv`.
`turbo`, having calculated a task hash prior to executing the `build` script, will be unable to discover
the `NEXT_PUBLIC_GA_ID` environment variable, and thus unable to partition the cache based
on its value. Be careful to ensure that all of your environment variables are configured prior to
invoking `turbo`!

Be careful to ensure that all of your environment variables are configured prior to invoking `turbo`!
```jsonc
{
"$schema": "https://turborepo.org/schema.json",
"pipeline": {
"build": {
"dependsOn": ["^build"],
"env": ["SOME_ENV_VAR"],
"outputs": [".next/**"],
},
}
}
```

### Using dotenv

Frameworks commonly use [dotenv][3] to automatically load environment variables when the dev server
starts or when creating a build. This makes it hard for Turborepo to understand the environment of your task by default:

- dotenv stores environment variables in a _file_ rather than in the environment
- this file is laoded _after_ Turborepo has already started execution of the task
- sometimes the file is git-ignored, so even if `inputs` are not specified, Turborepo won't be able to detect changes

To ensure you end up with the correct caching behavior for your task, add these `.env` files to the `globalDependencies` key:

```diff
{
"$schema": "https://turborepo.org/schema.json",
+ "globalDependencies": ["**/.env.*local"],
"pipeline": {
"build": {
"dependsOn": ["^build"],
"env": ["SOME_ENV_VAR"],
"outputs": [".next/**"],
},
}
}
```

Alternatively, you can add specific environment variables to the `inputs` key for specific tasks.

## Force overwrite cache

Expand Down Expand Up @@ -415,3 +456,4 @@ The hash of a given task is injected at execution time as an environment variabl

[1]: https://nextjs.org/docs/basic-features/environment-variables#exposing-environment-variables-to-the-browser
[2]: https://create-react-app.dev/docs/adding-custom-environment-variables/
[3]: https://github.com/motdotla/dotenv
1 change: 1 addition & 0 deletions examples/basic/turbo.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"$schema": "https://turbo.build/schema.json",
"globalDependencies": ["**/.env.*local"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mehulkar
Hi
I found that this setting seems to slow down tasks very much.

Tested with turbo 1.9.6 with basic example:

pnpm dlx create-turbo@1.9.6
? Where would you like to create your turborepo? ./my-turborepo
? Which package manager do you want to use? pnpm

Then, run pnpm lint (with cache == full turbo)
It takes 300-400ms.

pnpm lint

(omit)

 Tasks:    3 successful, 3 total
Cached:    3 cached, 3 total
  Time:    366ms >>> FULL TURBO

If I remove the globalDependencies setting:

pnpm lint

(omit)

 Tasks:    3 successful, 3 total
Cached:    3 cached, 3 total
  Time:    55ms >>> FULL TURBO

It is turbo!

If I make the following changes, it is also very fast.
(As much as when I delete globalDependencies.)

-  "globalDependencies": ["**/.env.*local"],
+  "globalDependencies": ["apps/**/.env.*local"],

So I think it may be better to limit the scope of glob.
What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it’s not a great solution for exactly this reason. You can specify all known files yourself or use a more limited glob.

@nathanhammond is also working on a feature to make this a top level configuration

"pipeline": {
"build": {
"outputs": [".next/**"]
Expand Down
1 change: 1 addition & 0 deletions examples/kitchen-sink/turbo.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"$schema": "https://turbo.build/schema.json",
"globalDependencies": ["**/.env.*local"],
"pipeline": {
"build": {
"outputs": [
Expand Down
1 change: 1 addition & 0 deletions examples/with-changesets/turbo.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"$schema": "https://turbo.build/schema.json",
"globalDependencies": ["**/.env.*local"],
"pipeline": {
"build": {
"outputs": ["dist/**", ".next/**"],
Expand Down
1 change: 1 addition & 0 deletions examples/with-docker/turbo.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"$schema": "https://turbo.build/schema.json",
"globalDependencies": ["**/.env.*local"],
"pipeline": {
"build": {
"outputs": ["dist/**", ".next/**", "public/dist/**"],
Expand Down
1 change: 1 addition & 0 deletions examples/with-npm/turbo.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"$schema": "https://turbo.build/schema.json",
"globalDependencies": ["**/.env.*local"],
"pipeline": {
"build": {
"dependsOn": ["^build"],
Expand Down
1 change: 1 addition & 0 deletions examples/with-prisma/turbo.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"$schema": "https://turbo.build/schema.json",
"globalDependencies": ["**/.env.*local"],
"globalEnv": ["NODE_ENV"],
"pipeline": {
"build": {
Expand Down
1 change: 1 addition & 0 deletions examples/with-react-native-web/turbo.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"$schema": "https://turbo.build/schema.json",
"globalDependencies": ["**/.env.*local"],
"pipeline": {
"build": {
"outputs": ["dist/**", ".next/**"],
Expand Down
1 change: 1 addition & 0 deletions examples/with-tailwind/turbo.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"$schema": "https://turbo.build/schema.json",
"globalDependencies": ["**/.env.*local"],
"pipeline": {
"build": {
"dependsOn": ["^build"],
Expand Down
1 change: 1 addition & 0 deletions packages/create-turbo/templates/_shared_ts/turbo.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"$schema": "https://turbo.build/schema.json",
"globalDependencies": ["**/.env.*local"],
"pipeline": {
"build": {
"dependsOn": ["^build"],
Expand Down