This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/vercel/turbo-site/E23SRQgKjGBmUfXXtnnVSMP97DCX
✅ Preview: https://turbo-site-git-gsoltis-scopeandsince.vercel.sh

probably add --deps flag as the truthy boolean

Going through implementation, I think we end up in the following situation:

1. User specifies --since or --scope. We now include dependencies and not dependents unless otherwise requested
2. No --scope or --since specified, we will build everything, making dependencies and dependents irrelevant.

This means that:

1. --include-dependencies is now the default, and does not ever need to be specified
2. --no-deps will never have an effect. Either we're building everything, or we've already implied --no-deps
3. A user might want to --include-dependents to facilitate the bottom-up build of an updated library and its dependents

Are there scenarios where we might want --no-dependencies? This would allow a build w/ --since or --scope to avoid building dependencies, but theoretically if they are dependencies, you need them to build...

Sharing some thoughts there on past conversations with @jaredpalmer:

Turbo should always run what it needs to accomplish a requested task. A task can be "requested" in three different ways, --since, --scope, or no args.

turbo run test # test for every package is being requested
turbo run test --since main # test for every package that has changed since `main` is being requested
turbo run test --scope=foo # test for the package foo is being requested

So in the first case the expectation is that test runs for every package and so any dependent tasks (i.e., to run test on an app you might need to run build on a dependent library first). Topological sorting is still expected for tasks to run properly.

For the second case, the same logic applies, but instead of running test for all packages, the ones that have changed since main are considered. In this case, "changed" means changes directly to the package or any of its dependencies. For example, if a library changed but an app didn't and the app depends on the library such that a dependent task (e.g., lib#build) would change the app's task (e.g., app#test) it's still expected for those to run.

For the third case, then foo#test would run and before that any dependent tasks.

If for some reason I want to target a task without including its dependencies, I would reach for turbo run test --no-deps where "deps" here is short for "dependencies", meaning that I don't want to run dependency tasks. I think right now --no-deps is understood to be no "dependents" which is the opposite. Perhaps not using the word "deps" and using "dependents" or "dependencies" would be clearer.

And finally, if I did want to include dependent tasks then I would go for turbo run build --scope=lib --include-dependents.

Something interesting to note is that the --no-deps and --include-dependencies seems to be a bit overloaded/confusing. It's not clear if the edge here is a dependency between packages or a dependency between tasks. For example turbo run build --scope=lib with an implicit --include-dependencies means that we want all transitive dependent tasks of lib#build. But turbo run build --scope=lib --include-dependents might mean I want to run the build task in all packages that depend on lib#build but not necessarily all dependent tasks. For example, app#test might depend on lib#build but I just want to run app#build.

Tools like pnpm use their syntax so model dependencies across packages since it's a package manager. But since Turbo is doing something lower level, dependencies between tasks and other arbitrary inputs (e.g., files in filesystem or env vars), the syntax might need to be about dependencies across tasks, not packages. For people that are used to Lerna or pnpm, this might be a bit of an issue as they search for capabilities or running a specific task (i.e., build) across packages. Perhaps there could be an option in the CLI to do that (maybe turbo run build --scope=lib --include-dependent-packages?) but I would say it's more important to get the DX for task dependencies right first.

Would it be helpful if we split run and run-many + filter/scope into different commands?

Would it be helpful if we split run and run-many + filter/scope into different commands?

Hmm, it would be nice if there was a single run command. Just to make sure that I'm on the same page with y'all, what's the expected behavior of --filter and --scope? Is one meant to act at the task level and the other at the package level? Because that might be what's causing some confusion for me, the --scope seems to be a parameter to filter packages but other options like --no-deps act at the task dependency level, not the package dependency level.

A detailed list of the relevant options and whether they're designed to act on the package graph or the task graph would be super useful!

@migueloller agreed that there's confusion between task- and package-level dependencies. I think the particular change in this PR is not making that better, so we'll close it.

What do you think about turbo run <task> --filter="some filter" being used to select a set of packages to apply a particular task to, and then turbo would do the work based on the pipeline configuration to figure out dependencies that need to run first? We could then defer to the filter syntax for whether or not to, for example, apply "test" to all dependencies or dependents of a package.

@gsoltis, so would --filter receive a package name like --scope does right now? Or would --filter replace --scope?

As far as distinguishing between tasks and packages, here's some thoughts:

• Running Turbo requires a task, so that could be the "entrypoint" (i.e., turbo run some-task).
• Once you've got an "entrypoint" task, then from there there's an entire task graph based on the pipelines configuration and the package graph.
• The desirable default behavior in my opinion is that from the "entrypoint" task all transitive dependencies of the task graph are run in topological order.
• In addition to this default behavior, it would be nice to be able to isolate the task, i.e., turbo run build --only --scope=foo (or some other option other than --only) is equivalent to running yarn workspace foo build if the repo uses Yarn workspaces. If --scope isn't used then it will just run all build tasks but not the dependencies.
• Another behavior is running the dependent tasks in the graph. For example, I changed so library and in CI I want to run affected tasks to make sure those aren't broken. More specifically, a libraries' build task which is depended on by an app's build task which itself is a dependency of the app's test task would all trigger if the library's build task is run with this behavior enabled.
• The previous behavior probably covers most cases of running dependent tasks, but it might be annoying when all you want is a subset of the tasks. For example, I want to build my library and any dependents, but the fact that test and deploy, etc. all depend on build doesn't mean I want those too. Perhaps when running dependents the behavior is more conservative and simply runs tasks with the same name (i.e., build across all dependent packages). Or perhaps this is opt-in behavior. e.g., turbo run build --scope=foo --no-dependencies --dependents --packages could mean that we care only about foo#build (whereas no --scope would mean our "entrypoint" tasks are all the build tasks), --no-dependencies means to not run the tasks that foo#build depends on, --dependents (the opposite of --no-dependents, which in this case is the default) means that we want dependent tasks in the graph to be included, and finally --packages (for lack of a better name, this one's horrible) means that what you're trying to run when you say --dependents is not the tasks that depend on foo#build and its transitive dependents, but instead run build for the packages that depend on foo.

Anyhow, that last example is a bit contrived. It's possible that there is no need to handle package dependents. I think that having an example repo where we can explore actual use cases would be a great way to explore what is and isn't needed and what behaviors feel most intuitive.

@migueloller The idea is that --filter would replace --scope and --since, and probably --ignore as well. The syntax (discussed here) is the PNPM syntax. This allows for things like "and all of my dependencies", "and all of my dependents", "has changed since git ref", as well as much more. Note that "dependencies" and "dependents" here are package dependencies, not tasks.

As you say, there would be an entrypoint that defines the task to be run, turbo run some-task --filter="some filter". Some filter would select some set of packages to have "some-task" applied.

So, in the case of:

1. test an application, turbo run test --filter="my-app" would use the pipeline configuration to determine the task-dependencies for my-app#test which likely includes "^build", causing all package-dependencies to get built
2. test things that package-depend on a library, turbo run test --filter="...my-lib" would start with <TASK>#test where TASK is my-lib and everything that transitively package-depends on my-lib. Then use the pipeline config to fill in task-dependencies that needs to happen. An extra bit of complication here is that a pipeline could configure a task-dependency from some other package to either my-lib#build or my-lib#test that would not necessarily show up in a package.json. TBD on what exactly to run in that scenario, although we could maybe consider it a configuration error to not have that dependency at a package level and still expect those tasks to be run.

I think the first example would be the equivalent of running a task in one package, as long as that task didn't have any task-dependencies. I'm not sure what the scenario would be to run only a specific task without running its task-dependencies first, but if you have a use case in mind, we can take that into account.

I think your last scenario is covered by using package-dependency and task-equality to determine what gets run in the library case. turbo run test --filter="...my-lib" would not trigger a publish in a package-dependent library because only the test task is being applied, and any of its task-dependencies.

Pretty excited about the --filter flag. I think it will streamline and make the --scope, --ignore, and --since combination a lot less confusing! Can't wait for it :)

@gsoltis, just went over the RFC. It looks great!

I think you're right that there probably isn't a case where you want to run a task but don't want to run its dependencies. I looked around in my repo and the only places that I avoided to use --include-dependencies was when I had just ran it with --no-deps and --include-dependencies to work around the current issues.

Something interesting about this RFC is that it could be expanded to support filtering the task graph. i.e., --package-filter could work the exact same way as --filter in the RFC and --task-filter also, but tasks are identified as they are in the pipeline config (<task>#<package>). That being said, there needs to be a use case for filtering the task graph that can't be achieved from filtering the package graph to introduce an extra mode of filtering like this, I think. Given the previous statement that you never (?) want to run a task without its dependencies then perhaps the only "input" for task filtering that's needed is the task name provided to the command (i.e., build in turbo run build).