extensions:pip.bzl¶

pip module extension for use with bzlmod

pip¶

This extension is used to make dependencies from pip available.

pip.parse: To use, call pip.parse() and specify hub_name and your requirements file. Dependencies will be downloaded and made available in a repo named after the hub_name argument.

Each pip.parse() call configures a particular Python version. Multiple calls can be made to configure different Python versions, and will be grouped by the hub_name argument. This allows the same logical name, e.g. @pip//numpy to automatically resolve to different, Python version-specific, libraries.

pip.whl_mods: This tag class is used to help create JSON files to describe modifications to the BUILD files for wheels.

pip.override¶

pip.override(file, patch_strip=0, patches)

Apply any overrides (e.g. patches) to a given Python distribution defined by other tags in this extension.

ATTRIBUTES ¶

file¶:: (required string) The Python distribution file name which needs to be patched. This will be applied to all repositories that setup this distribution via the pip.parse tag class.
patch_strip¶:: (optional [int][int], default 0) The number of leading path segments to be stripped from the file name in the patches.
patches¶:: (required list of labels) A list of patches to apply to the repository after ‘whl_library’ is extracted and BUILD.bazel file is generated.

pip.parse¶

pip.parse(hub_name, python_version, whl_modifications={}, annotations={}, requirements_darwin=None, requirements_linux=None, requirements_lock=None, requirements_windows=None, download_only=False, enable_implicit_namespace_pkgs=False, environment={}, envsubst=[], experimental_requirement_cycles={}, experimental_target_platforms=[], extra_pip_args=[], isolated=True, pip_data_exclude=[], python_interpreter=””, python_interpreter_target=None, quiet=True, timeout=600)

This tag class is used to create a pip hub and all of the spokes that are part of that hub. This tag class reuses most of the pip attributes that are found in @rules_python//python/pip_install:pip_repository.bzl. The exception is it does not use the arg ‘repo_prefix’. We set the repository prefix for the user and the alias arg is always True in bzlmod.

ATTRIBUTES ¶

hub_name¶:

(required string) The name of the repo pip dependencies will be accessible from.

This name must be unique between modules; unless your module is guaranteed to always be the root module, it’s highly recommended to include your module name in the hub name. Repo mapping, use_repo(..., pip="my_modules_pip_deps"), can be used for shorter local names within your module.

Within a module, the same hub_name can be specified to group different Python versions of pip dependencies under one repository name. This allows using a Python version-agnostic name when referring to pip dependencies; the correct version will be automatically selected.

Typically, a module will only have a single hub of pip dependencies, but this is not required. Each hub is a separate resolution of pip dependencies. This means if different programs need different versions of some library, separate hubs can be created, and each program can use its respective hub’s targets. Targets from different hubs should not be used together.

python_version¶:

(required string) The Python version the dependencies are targetting, in Major.Minor format (e.g., “3.11”). Patch level granularity (e.g. “3.11.1”) is not supported. If not specified, then the default Python version (as set by the root module or rules_python) will be used.

If an interpreter isn’t explicitly provided (using python_interpreter or python_interpreter_target), then the version specified here must have a corresponding python.toolchain() configured.

whl_modifications¶:

(optional dict of label to string, default {}) A dict of labels to wheel names that is typically generated by the whl_modifications. The labels are JSON config files describing the modifications.

annotations¶:

(optional dict of string to string, default {}) Optional annotations to apply to packages

requirements_darwin¶:

(optional label, default None) Override the requirements_lock attribute when the host platform is Mac OS

requirements_linux¶:

(optional label, default None) Override the requirements_lock attribute when the host platform is Linux

requirements_lock¶:

(optional label, default None) A fully resolved ‘requirements.txt’ pip requirement file containing the transitive set of your dependencies. If this file is passed instead of ‘requirements’ no resolve will take place and pip_repository will create individual repositories for each of your dependencies so that wheels are fetched/built only for the targets specified by ‘build/run/test’. Note that if your lockfile is platform-dependent, you can use the requirements_[platform] attributes.

requirements_windows¶:

(optional label, default None) Override the requirements_lock attribute when the host platform is Windows

download_only¶:

(optional [bool][bool], default False) Whether to use “pip download” instead of “pip wheel”. Disables building wheels from source, but allows use of –platform, –python-version, –implementation, and –abi in –extra_pip_args to download wheels for a different platform from the host platform.

enable_implicit_namespace_pkgs¶:

(optional [bool][bool], default False) If true, disables conversion of native namespace packages into pkg-util style namespace packages. When set all py_binary and py_test targets must specify either legacy_create_init=False or the global Bazel option --incompatible_default_to_explicit_init_py to prevent __init__.py being automatically generated in every directory.

This option is required to support some packages which cannot handle the conversion to pkg-util style.

environment¶:

(optional dict of string to string, default {}) Environment variables to set in the pip subprocess. Can be used to set common variables such as http_proxy, https_proxy and no_proxy Note that pip is run with “–isolated” on the CLI so PIP_<VAR>_<NAME> style env vars are ignored, but env vars that control requests and urllib3 can be passed. If you need PIP_<VAR>_<NAME>, take a look at extra_pip_args and envsubst.

envsubst¶:

(optional list of string, default []) A list of environment variables to substitute (e.g. ["PIP_INDEX_URL", "PIP_RETRIES"]). The corresponding variables are expanded in extra_pip_args using the syntax $VARNAME or ${VARNAME} (expanding to empty string if unset) or ${VARNAME:-default} (expanding to default if the variable is unset or empty in the environment). Note: On Bazel 6 and Bazel 7 changes to the variables named here do not cause packages to be re-fetched. Don’t fetch different things based on the value of these variables.

experimental_requirement_cycles¶:

(optional dict of string to list of string, default {}) A mapping of dependency cycle names to a list of requirements which form that cycle.

Requirements which form cycles will be installed together and taken as dependencies together in order to ensure that the cycle is always satisified.

Example: sphinx depends on sphinxcontrib-serializinghtml When listing both as requirements, ala

py_binary(
  name = "doctool",
  ...
  deps = [
    "@pypi//sphinx:pkg",
    "@pypi//sphinxcontrib_serializinghtml",
   ]
)

Will produce a Bazel error such as

ERROR: .../external/pypi_sphinxcontrib_serializinghtml/BUILD.bazel:44:6: in alias rule @pypi_sphinxcontrib_serializinghtml//:pkg: cycle in dependency graph:
    //:doctool (...)
    @pypi//sphinxcontrib_serializinghtml:pkg (...)
.-> @pypi_sphinxcontrib_serializinghtml//:pkg (...)
|   @pypi_sphinxcontrib_serializinghtml//:_pkg (...)
|   @pypi_sphinx//:pkg (...)
|   @pypi_sphinx//:_pkg (...)
`-- @pypi_sphinxcontrib_serializinghtml//:pkg (...)

Which we can resolve by configuring these two requirements to be installed together as a cycle

pip_parse(
  ...
  experimental_requirement_cycles = {
    "sphinx": [
      "sphinx",
      "sphinxcontrib-serializinghtml",
    ]
  },
)

Warning: If a dependency participates in multiple cycles, all of those cycles must be collapsed down to one. For instance a <-> b and a <-> c cannot be listed as two separate cycles.

experimental_target_platforms¶:

(optional list of string, default []) A list of platforms that we will generate the conditional dependency graph for cross platform wheels by parsing the wheel metadata. This will generate the correct dependencies for packages like sphinx or pylint, which include colorama when installed and used on Windows platforms.

An empty list means falling back to the legacy behaviour where the host platform is the target platform.

WARNING: It may not work as expected in cases where the python interpreter implementation that is being used at runtime is different between different platforms. This has been tested for CPython only.

For specific target platforms use values of the form <os>_<arch> where <os> is one of linux, osx, windows and arch is one of x86_64, x86_32, aarch64, s390x and ppc64le.

You can also target a specific Python version by using cp3<minor_version>_<os>_<arch>. If multiple python versions are specified as target platforms, then select statements of the lib and whl targets will include usage of version aware toolchain config settings like @rules_python//python/config_settings:is_python_3.y.

Special values: host (for generating deps for the host platform only) and <prefix>_* values. For example, cp39_*, linux_*, cp39_linux_*.

NOTE: this is not for cross-compiling Python wheels but rather for parsing the whl METADATA correctly.

extra_pip_args¶:

(optional list of string, default []) Extra arguments to pass on to pip. Must not contain spaces.

Supports environment variables using the syntax $VARNAME or ${VARNAME} (expanding to empty string if unset) or ${VARNAME:-default} (expanding to default if the variable is unset or empty in the environment), if "VARNAME" is listed in the envsubst attribute. See also envsubst.

isolated¶:

(optional [bool][bool], default True) Whether or not to pass the –isolated flag to the underlying pip command. Alternatively, the RULES_PYTHON_PIP_ISOLATED environment variable can be used to control this flag.

pip_data_exclude¶:

(optional list of string, default []) Additional data exclusion parameters to add to the pip packages BUILD file.

python_interpreter¶:

(optional string, default "") The python interpreter to use. This can either be an absolute path or the name of a binary found on the host’s PATH environment variable. If no value is set python3 is defaulted for Unix systems and python.exe for Windows.

python_interpreter_target¶:

(optional label, default None) If you are using a custom python interpreter built by another repository rule, use this attribute to specify its BUILD target. This allows pip_repository to invoke pip using the same interpreter as your toolchain. If set, takes precedence over python_interpreter. An example value: “@python3_x86_64-unknown-linux-gnu//:python”.

quiet¶:

(optional [bool][bool], default True) If True, suppress printing stdout and stderr output to the terminal.

timeout¶:

(optional [int][int], default 600) Timeout (in seconds) on the rule’s execution duration.

pip.whl_mods¶

pip.whl_mods(additive_build_content=””, additive_build_content_file=None, copy_executables={}, copy_files={}, data=[], data_exclude_glob=[], hub_name, srcs_exclude_glob=[], whl_name)

This tag class is used to create JSON file that are used when calling wheel_builder.py. These JSON files contain instructions on how to modify a wheel’s project. Each of the attributes create different modifications based on the type of attribute. Previously to bzlmod these JSON files where referred to as annotations, and were renamed to whl_modifications in this extension.

ATTRIBUTES ¶

additive_build_content¶:

(optional string, default "") (str, optional): Raw text to add to the generated BUILD file of a package.

additive_build_content_file¶:

(optional label, default None) (label, optional): path to a BUILD file to add to the generated BUILD file of a package. You cannot use both additive_build_content and additive_build_content_file arguments at the same time.

copy_executables¶:

(optional dict of string to string, default {}) (dict, optional): A mapping of src and out files for [@bazel_skylib//rules:copy_file.bzl][cf]. Targets generated here will also be flagged as executable.

copy_files¶:

(optional dict of string to string, default {}) (dict, optional): A mapping of src and out files for [@bazel_skylib//rules:copy_file.bzl][cf]

data¶:

(optional list of string, default []) (list, optional): A list of labels to add as data dependencies to the generated py_library target.

data_exclude_glob¶:

(optional list of string, default []) (list, optional): A list of exclude glob patterns to add as data to the generated py_library target.

hub_name¶:

(required string) Name of the whl modification, hub we use this name to set the modifications for pip.parse. If you have different pip hubs you can use a different name, otherwise it is best practice to just use one.

You cannot have the same hub_name in different modules. You can reuse the same name in the same module for different wheels that you put in the same hub, but you cannot have a child module that uses the same hub_name.

srcs_exclude_glob¶:

(optional list of string, default []) (list, optional): A list of labels to add as srcs to the generated py_library target.

whl_name¶:

(required string) The whl name that the modifications are used for.