-
Notifications
You must be signed in to change notification settings - Fork 915
Open
Description
Currently, TFLM's Bazel build uses the system Python installation, which can lead to reproducibility issues and dependency conflicts. A hermetic Python toolchain would ensure all developers and CI use the exact same Python version and environment.
Current Problems
When system Python dependencies mix with Bazel's pip repositories, we see issues like:
- Version conflicts: System-installed packages (e.g.,
numpy
,protobuf
) can conflict with versions specified inpython_requirements.txt
- Tool inconsistency: Developers may have different versions of
yapf
,cpplint
, or other tools installed globally - Hidden dependencies: Builds may accidentally depend on system packages not declared in requirements
- "Works on my machine": Builds succeed locally but fail in CI or on other developers' machines due to different system Python setups
Proposed Solution
Use rules_python's hermetic toolchain support. Something like:
# In WORKSPACE
load("@rules_python//python:repositories.bzl", "python_register_toolchains")
python_register_toolchains(
name = "python",
python_version = "3.10", # Pin to specific version
)
# Pip dependencies fully isolated from system
load("@rules_python//python:pip.bzl", "pip_parse")
pip_parse(
name = "pip_deps",
requirements_lock = "//third_party:python_requirements.txt",
python_interpreter_target = "@python//:python", # Use hermetic Python
)
This would yield:
- Bazel downloads and manages its own Python interpreter
- All pip dependencies are isolated from system packages
- Same behavior across all platforms and environments
This has been wished for on several occasions: #2027, #2743, #3126.
Metadata
Metadata
Assignees
Labels
No labels