First brought up in this question: #2005

Context

It is a common practice for library developers to create a distinction between runtime dependencies, and build dependencies by using dependencies (for runtime dependencies) and devDependencies (for build dependencies).

Problem

At the time, there isn't a way to express that distinction in the turbo.json file's dependsOn config.

The cost is that caches are unnecessarily purged in cases where a package's dependency changes, even though the dependency is not referenced during the build, and is only referenced during the runtime of the package.

Example

package-1 > input

import someDep from 'package-2';
console.log(someDep);

package-1 > output

/* 
Maybe here some other things are added by the build, 
but `someDep` reference is intact because this is a runtime dependency
*/
import someDep from 'package-2';
console.log(someDep);

package-1/package.json

{
  "name": "package-1",
  "scripts": {
    "build": "some-internal-build-tool build" // << USE dev dependency
  },
  "dependencies": {
    "package-2": "^1.0.0"
  },
  "devDependencies": {
    "some-internal-build-tool": "^1.0.0"
  }
}

package-2/package.json

{
  "name": "package-2",
  "scripts": {
    "build": "some-internal-build-tool build" // << USE dev dependency
  },
  "devDependencies": {
    "some-internal-build-tool": "^1.0.0"
  }
}

some-internal-build-tool/package.json

{
  "name": "some-internal-build-tool",
  "scripts": {
    "build": "some-other-open-source-tool build" // << Uses external tool here
  },
  "devDependencies": {
    "some-other-open-source-tool ": "^1.0.0"
  }
}

turbo.json

{
  "$schema": "https://turborepo.org/schema.json",
  "pipeline": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": ["lib/**"],
      "inputs": ["src/**"]
    }
  }
}

Then

If I make a change in package-2, I don't need to recompile package-1 because this is a runtime dependency. However, if I make a change to some-internal-build-tool then I should need to recompile package-1 and package-2.

Proposal (Idea)

Allow developers to isolate dependencies and devDependencies in the upstream reference. This will allow us to continue maintaining a distinction between runtime and build dependencies (Using dependencies, and devDependencies as proxies, respectively), while optimizing cache hits.

Syntax

I don't know what the preference would be, but seems like it may make sense to just build on top of the convention to use an arbitrary symbol as a prefix (for upstream) and suffix (for downstream) [as ^dep or dep^ is used currently], but the symbol should be unique.

Additionally, it may make sense to suggest a more verbose but clear syntax to denote the dependency group reference

{
  "dependsOn": [
    {"input": "devDependencies", "target": "build"}
  ]
}

Final thoughts

I think allowing this distinction is a reasonable step forward in allowing developers more expressive freedom to optimize cache hits.

However, I recognize that this is still limited and leads to unnecessary cache misses since there isn't a way to provide granular dependencies for a script. This a product of the turbo.json's global nature, and the fact that turbo assumes that all dependencies (including devDependencies) can affect the output of a given command.

A solution could be to allow providing granular dependencies per project/script (Like Bazel does with BUILD files).

For example, lets say 2 dependencies affect the result of build, and 2 other dependencies affect the result of test

{
  "scripts": {
    "build": "some-command",
    "test": "some-other-command"
  },
  // Maybe allow this in a package.json file
  "turbo": {
    "dependencies": {
      "build": [
        "pkg-1 build",
        "pkg-2 build"
      ],
      "test": [
        "pkg-3 build",
        "pkg-4 build"
      ]
    }
  },
  "devDependencies": {
    "pkg-1": "0.0.0",
    "pkg-2": "0.0.0"
    "pkg-3": "0.0.0",
    "pkg-4": "0.0.0",
  }
}

So my proposal is likely not the best solution, but a maybe reasonable step forward?