feat(pypi): support importing uv.lock file by aignas · Pull Request #3785 · bazel-contrib/rules_python

aignas · 2026-05-16T17:07:35Z

Part of this is vibe coded, but I thought that the approach might have been rigorous
enough to submit a PR.

The strategy was:

First add a way for us to create a uv.lock file from the lock rule.
Then add a uv.lock file to JSON converter.
Then add a way to read the uv.lock file together with the requirements file
and verify things are OK.
Reuse most of the code.

Extra things that we could do:

Full test suite for various uv.lock scenarios and ensure parity with
requirements.txt files.
Call the PyPI index to understand if the packages are yanked or not - lock
file does not have that information.
Read the pyproject.toml file to get the index values for each package.

Summary:

feat(pypi): add uv.lock parsing support to parse_requirements
test(pypi): add tests for uv.lock parsing in parse_requirements
test(uv): add lock rule integration tests for uv.lock format
test: add uv_pypi end-to-end integration test
docs: add uv.lock documentation and sample files

Closes #3557
Work towards #2787

gemini-code-assist

Code Review

This pull request adds support for uv.lock files in rules_python, introducing a toml2json conversion utility and updating the lock rule and requirement parsing logic. The review identifies critical compatibility issues, specifically the toml2json tool's reliance on Python 3.11's tomllib and missing serialization for date/time objects. Furthermore, the feedback highlights logic errors in how package extras are handled—which could lead to dependency bloat or failed consistency checks—and suggests improvements for platform resolution and path handling in shell scripts.

gemini-code-assist · 2026-05-16T17:12:54Z

+    """Parse requirements using uv.lock as the primary source."""
+    ret = _parse_uv_lock_json(
+        uv_lock_json = uv_lock_json,
+        all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else [],


If requirements_by_platform is empty (which is common when using uv.lock as the primary source), all_platforms will be an empty list. This results in all packages having empty target_platforms, which will likely cause issues in downstream rules that expect platform information for wheel selection.

Suggested change

all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else [],

all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else sorted(platforms.keys()),

gemini-code-assist · 2026-05-16T17:12:55Z

+        for extra in pkg.get("provides-extras", pkg.get("extras", [])):
+            if extra not in entry["extras"]:
+                entry["extras"][extra] = None


Including all provides-extras in the requirement line for every package is likely incorrect. provides-extras lists all extras a package defines, not necessarily what was resolved or requested. Including all of them will force the installation of all optional dependencies for every package in the lock file, leading to significant dependency bloat. It might be better to omit extras from the requirement line if the lock file already provides the specific version and URL, or only include the extras that were part of the resolution.

gemini-code-assist · 2026-05-16T17:12:55Z

+        for dep in pkg.get("dependencies", []):
+            extra = dep.get("extra")
+            if extra and extra not in uv_packages[name]["extras"]:
+                uv_packages[name]["extras"][extra] = None


The logic for collecting extras in the consistency check appears to be flawed. It iterates over the dependencies of pkg and adds the extra name to uv_packages[name]["extras"], where name is the name of the requiring package. Later, when checking a package item, it looks for extras in uv_packages[item.name]["extras"]. This will fail to find extras that were requested by other packages.

- tomllib: try/except fallback to tomli for Python <3.11 - json_serializer: add datetime.date and datetime.time support - all_platforms: use sorted(platforms.keys()) fallback - $PWD/ path: check if python_path is absolute - extra_pip_args: pass through to _parse_uv_lock_json - Add uv_lock tests: multiple packages, extra_pip_args, multi-os - Update plan.md with review cycle instructions

aignas · 2026-05-16T18:33:12Z

Addressed all review comments. Please re-review when you get a chance. @gemini-code-assist

gemini-code-assist

Code Review

This pull request introduces support for uv.lock files in rules_python, adding a new lock_format attribute to the lock rule and a toml2json utility for parsing. It also implements a consistency check between uv.lock and requirements files. The reviewer identified a potential runtime error in the uv.lock parser due to unsafe dictionary access for optional hash keys and suggested simplifying redundant logic during hash extraction.

- Add python_3_14 toolchain in MODULE.bazel - Always use Python 3.14 interpreter for uv.lock JSON conversion - Remove tomli fallback from toml2json.py (Python 3.14 has tomllib) - Simplify convert_uv_lock_to_json to use fixed interpreter

- Add bazel-in-bazel integration test under tests/integration/uv_lock/ - Test converts uv.lock TOML to JSON using toml2json tool - Remove dead uv_pypi test data (placeholder only) - Restore tomli fallback in toml2json.py for backward compatibility

aignas · 2026-05-16T18:43:40Z

Addressed all review comments and added Python 3.14 toolchain + integration test. Please re-review. @gemini-code-assist

gemini-code-assist

Code Review

This pull request introduces support for uv.lock files as a primary metadata source in rules_python. Key additions include a toml2json tool for parsing, a new uv_lock format for the lock rule, and consistency checks within parse_requirements. Reviewer feedback suggests adopting more idiomatic Starlark dictionary access, implementing safer hash prefix stripping, and improving the portability of shell commands by replacing hardcoded environment variables with Bazel-provided paths.

jvolkman · 2026-05-17T14:39:43Z

The latest release of toml.bzl makes bazel_lib a dev dependency. It now uses skylib for the bzl_library rules.

Remove the old toml2json Python tool and uv_lock.bzl in favor of the pure Starlark toml.bzl decoder. Update tests to pass toml_decode mock, remove is_rules_python_root references, and fix virtual package test expectations. Clean up BUILD.bazel files that referenced deleted targets.

aignas · 2026-05-19T03:54:01Z

            ),
            logger = logger,
        ),
+        uv_lock = pip_attr.uv_lock,


Add a test to ensure this is tested.

… lock support Remove the lock_format attribute and detect whether to use uv lock or uv pip compile from the output file extension (.lock = uv lock, else requirements). Add a Windows bat template for uv lock support. The python interpreter is passed through --python flag consistently.

rickeylev

Overall LGTM. Some minor questions and nits. But overall, this fit in rather nicely with the pipstar code you've written, nice!

Require a strict `uv.lock` suffix for uv lock targets to disambiguate from pip-compile outputs, and fix absolute Python interpreter path evaluation during lockfile generation. Additionally, restore helpful structural comments inside requirement parsing logic and register missing TOML Stardoc dependencies.

rickeylev · 2026-06-07T07:04:26Z

(agent comment) Addressed the second review cycle comments (enforcing uv.lock suffix, renaming _template attribute, restoring requirement parsing comments, and registering missing Stardoc TOML dependencies). Please re-review @gemini-code-assist @rickeylev

gemini-code-assist

Code Review

This pull request implements support for uv.lock files in rules_python, allowing them to be used as the primary source for package metadata and introducing a rule to generate them. The feedback identifies several issues: a potential path resolution failure in lock.bzl when the Python interpreter path is absolute, an undefined repository error caused by exposing virtual packages with no sources in the hub, and a version mismatch for the toml.bzl dependency between MODULE.bazel and py_repositories.bzl.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

gemini-code-assist · 2026-06-07T07:07:02Z

+        item = struct(
+            name = norm_name,
+            is_exposed = True,
+            is_multiple_versions = len(versions) > 1,
+            # TODO @aignas 2026-05-17: use the default index that is used in parsing the
+            # requirements if it is not known in the uv.lock file. We need to get this from the
+            # pyproject.toml file uv.tool configuration.
+            index_url = "",
+            srcs = pkg_srcs,
+        )


Setting is_exposed = True for virtual packages (which have no sources, i.e., srcs = []) causes the hub repository to expose them as targets. However, since they have no sources, no corresponding whl_library repository is created for them. This leads to undefined repository errors (e.g., @hub_virtual_pkg not found) when Bazel attempts to resolve or query the hub's targets. We should set is_exposed to len(pkg_srcs) > 0 so that virtual packages without sources are not exposed in the hub.

Suggested change

item = struct(

name = norm_name,

is_exposed = True,

is_multiple_versions = len(versions) > 1,

# TODO @aignas 2026-05-17: use the default index that is used in parsing the

# requirements if it is not known in the uv.lock file. We need to get this from the

# pyproject.toml file uv.tool configuration.

index_url = "",

srcs = pkg_srcs,

)

item = struct(

name = norm_name,

is_exposed = len(pkg_srcs) > 0,

is_multiple_versions = len(versions) > 1,

# TODO @aignas 2026-05-17: use the default index that is used in parsing the

# requirements if it is not known in the uv.lock file. We need to get this from the

# pyproject.toml file uv.tool configuration.

index_url = "",

srcs = pkg_srcs,

)

gemini-code-assist · 2026-06-07T07:07:02Z

+        struct(
+            name = "virtual_pkg",
+            index_url = "",
+            is_exposed = True,
+            is_multiple_versions = False,
+            srcs = [],
+        ),


Update the unit test expectation to match the corrected is_exposed behavior for virtual packages (which should be False since they have no sources).

Suggested change

struct(

name = "virtual_pkg",

index_url = "",

is_exposed = True,

is_multiple_versions = False,

srcs = [],

),

struct(

name = "virtual_pkg",

index_url = "",

is_exposed = False,

is_multiple_versions = False,

srcs = [],

)

Add dedicated parsing support for direct artifact URL sources in `uv.lock` files to ensure compatibility with non-registry dependencies. Implemented by adding a dedicated `_add_direct_url_entry` helper and associated unit test coverage in `parse_requirements_tests.bzl`.

Adopt `str.removeprefix()` for safe hash stripping, sync `toml.bzl` to 0.4.1 in `py_repositories.bzl`, add absolute interpreter path checks in `lock.bzl`, format `uv_lock` docstrings with `VERSION_NEXT_FEATURE`, and remove temporary Python 3.14 toolchains.

rickeylev · 2026-06-07T07:50:35Z

Addressed the remaining unresolved review threads (adopting str.removeprefix(), syncing toml.bzl to 0.4.1 in py_repositories.bzl, adding absolute path check in lock.bzl, formatting uv_lock docstrings with VERSION_NEXT_FEATURE, and removing temporary Python 3.14 toolchains). Please re-review @gemini-code-assist @rickeylev

gemini-code-assist

Code Review

This pull request introduces support for using uv.lock files as the primary source of package metadata in rules_python. It integrates a pure Starlark TOML decoder to parse uv.lock and extract packages, versions, extras, and sources (wheels, sdists, VCS, and direct URLs). Additionally, it updates the lock rule to support generating uv.lock files and adds comprehensive unit tests. The code review identified critical issues where the shell and batch scripts fail when copying uv.lock onto itself, a correctness bug where assigning all platforms to multiple versions of a package causes repository conflicts during hub building, and an issue where git revisions are not appended to VCS URLs, leading to unpinned dependencies.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

gemini-code-assist · 2026-06-07T07:52:35Z

+                distribution = info["distribution"],
+                extra_pip_args = extra_pip_args or [],
+                requirement_line = requirement_line,
+                target_platforms = list(all_platforms),


Setting target_platforms = list(all_platforms) for all package versions in _parse_uv_lock introduces a correctness bug when a package has multiple versions in uv.lock (e.g., due to different Python versions or environment markers).

Because all versions of the package are assigned the exact same target_platforms, they will all generate the same config settings in _add_whl_library. This causes a conflict during the hub building phase, resulting in a build failure: attempting to override an existing repo '...' for config setting '...' with a new repo '...'.

To fix this, we should parse and evaluate the resolution-markers (or package-level markers) for each package version against the configured platforms to determine the correct subset of target_platforms for each version. If no markers are present, we can default to all_platforms.

rickeylev

Fixed a variety of comments

# Conflicts: # MODULE.bazel # python/private/pypi/hub_builder.bzl # python/private/pypi/parse_requirements.bzl # python/uv/private/lock.bzl

…ts call Align `parse_requirements()` invocation in `hub_builder.bzl` with upstream marker evaluation refactoring.

Prevent wheel repository override collisions across incompatible Python versions in uv.lock by properly evaluating resolution-markers against target platforms.

…o aignas.feat.uv-lock

…ng upstream main

…o aignas.feat.uv-lock # Conflicts: # python/uv/private/lock.bzl

rickeylev · 2026-06-09T06:50:22Z

@aignas PTAL -- CI is happy! But the unresolved comments from Gemini I think might be valid?

aignas added 9 commits May 17, 2026 01:56

feat(pypi): add toml2json tool for converting uv.lock TOML to JSON

a58cf50

test(pypi): add tests for toml2json tool

363c141

feat(pypi): add uv_lock utility for uv.lock to JSON conversion

6f4e925

feat(uv): update lock rule with uv.lock JSON conversion support

29a7993

feat(pypi): add uv.lock parsing support to parse_requirements

a063d53

test(pypi): add tests for uv.lock parsing in parse_requirements

9fea66c

test(uv): add lock rule integration tests for uv.lock format

aa19ceb

test: add uv_pypi end-to-end integration test

ee069e4

docs: add uv.lock documentation and sample files

dd53e29

gemini-code-assist Bot reviewed May 16, 2026

View reviewed changes

aignas added 2 commits May 17, 2026 02:18

remove dead code and add plan.md for iterating

d5709ea

gemini-code-assist Bot reviewed May 16, 2026

View reviewed changes

Comment thread python/private/pypi/parse_requirements.bzl Outdated

Comment thread python/private/pypi/parse_requirements.bzl Outdated

aignas added 3 commits May 17, 2026 03:38

docs: update plan.md with completed stages

6db3345

gemini-code-assist Bot reviewed May 16, 2026

View reviewed changes

aignas added 2 commits May 18, 2026 22:11

wip

1f7b695

aignas commented May 19, 2026

View reviewed changes

aignas added 5 commits May 19, 2026 20:51

add deps for workspace

b46e0fd

fix the lock rule

be3012a

avoid eager loading of the hub repo

e3646a5

toml bump

9c1e30d

rickeylev approved these changes May 27, 2026

View reviewed changes

gemini-code-assist Bot reviewed Jun 7, 2026

View reviewed changes

rickeylev added 2 commits June 7, 2026 07:13

gemini-code-assist Bot reviewed Jun 7, 2026

View reviewed changes

rickeylev reviewed Jun 7, 2026

View reviewed changes

Comment thread python/private/py_repositories.bzl

Comment thread python/uv/private/lock.bzl Outdated

Comment thread python/private/pypi/extension.bzl Outdated

Comment thread python/private/pypi/parse_requirements.bzl Outdated

rickeylev added 2 commits June 8, 2026 06:09

Merge remote-tracking branch 'upstream/main' into aignas.feat.uv-lock

f8856c5

# Conflicts: # MODULE.bazel # python/private/pypi/hub_builder.bzl # python/private/pypi/parse_requirements.bzl # python/uv/private/lock.bzl

fix(pypi): remove obsolete evaluate_markers arg from parse_requiremen…

472d5d9

…ts call Align `parse_requirements()` invocation in `hub_builder.bzl` with upstream marker evaluation refactoring.

rickeylev force-pushed the aignas.feat.uv-lock branch from 0d55407 to 7c98eb2 Compare June 8, 2026 07:42

Filter target_platforms by resolution-markers in _parse_uv_lock

67c6b23

Prevent wheel repository override collisions across incompatible Python versions in uv.lock by properly evaluating resolution-markers against target platforms.

rickeylev force-pushed the aignas.feat.uv-lock branch from 7c98eb2 to 67c6b23 Compare June 8, 2026 07:45

rickeylev added 4 commits June 8, 2026 15:28

Merge branch 'main' of https://github.com/bazelbuild/rules_python int…

0701e53

…o aignas.feat.uv-lock

Update bzlmod_lockfile integration test MODULE.bazel.lock after mergi…

4efd940

…ng upstream main

Merge branch 'main' of https://github.com/bazelbuild/rules_python int…

bdba183

…o aignas.feat.uv-lock # Conflicts: # python/uv/private/lock.bzl

Append rev and subdirectory to VCS git source URLs in _add_vcs_entry

d855180

rickeylev marked this pull request as ready for review June 9, 2026 06:48

	all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else [],
	all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else sorted(platforms.keys()),

Uh oh!

Conversation

aignas commented May 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aignas commented May 16, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

aignas commented May 16, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jvolkman commented May 17, 2026

Uh oh!

Uh oh!

aignas May 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rickeylev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rickeylev commented Jun 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

gemini-code-assist Bot Jun 7, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rickeylev commented Jun 7, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

aignas commented May 16, 2026 •

edited

Loading

rickeylev commented Jun 7, 2026 •

edited

Loading