ci: require setup-matrix (except maven/nuget) + run live-API e2e on PR#103
Closed
Mikola Lysenko (mikolalysenko) wants to merge 1 commit into
Closed
ci: require setup-matrix (except maven/nuget) + run live-API e2e on PR#103Mikola Lysenko (mikolalysenko) wants to merge 1 commit into
Mikola Lysenko (mikolalysenko) wants to merge 1 commit into
Conversation
Remove the known_gap bypasses so the setup-flow matrix fails CI when an ecosystem's `setup` integration is broken, and enable the live-API e2e smoke suites as blocking PR checks. maven and nuget stay experimental. matrix.json: - Flip baseline_supported false -> true for poetry, pdm, cargo, bundler, go, composer, deno (single) and pip, uv (workspace). An unmet ideal now classifies as a blocking *regression* instead of a non-blocking known_gap. maven/mvn and nuget/dotnet remain baseline_supported:false. - Update the matrix comments to reflect that baseline_supported now marks "required to support setup" (true everywhere except maven/nuget). ci.yml: - Drop `continue-on-error: true` on the setup-matrix job so the orchestrator's non-zero exit (on regression) fails the job. - Add e2e_npm/pypi/gem/scan to the e2e PR matrix. They keep #[ignore] and are selected via `-- --ignored`; the existing Node/Python/Ruby setup steps already key off these suite names. These hit the real public proxy and may go red on upstream 503s rather than a regression; the hermetic e2e-docker matrix remains the always-on counterpart. Note: for these jobs to block merges they must be listed in the repo's required status checks (branch-protection UI). Underlying setup support for the now-required ecosystems is unchanged, so those legs are expected to go red until `setup` wires their install hooks. Assisted-by: Claude Code:opus-4-8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Removes the
known_gapbypasses so the setup-flow matrix fails CI when anecosystem's
setupintegration is broken, and enables the live-API e2esmoke suites as blocking PR checks. maven and nuget stay experimental.
tests/setup_matrix/matrix.jsonbaseline_supported: false → truefor poetry, pdm, cargo, bundler,go, composer, deno (single) and pip, uv (workspace). An unmet ideal
now classifies as a blocking
regressioninstead of a non-blockingknown_gap.maven/mvnandnuget/dotnetremainbaseline_supported: false.baseline_supportednow marks "required tosupport
setup" (true everywhere except maven/nuget)..github/workflows/ci.ymlcontinue-on-error: trueon thesetup-matrixjob so theorchestrator's non-zero exit (on regression) fails the job.
e2e_npm/e2e_pypi/e2e_gem/e2e_scanto thee2ePR matrix.They keep
#[ignore]and are selected via-- --ignored; the existingNode/Python/Ruby setup steps already key off these suite names.
Expected effects / caveats
setupactually wires the hooks for thenow-required ecosystems (cargo, gem, golang, composer, deno, and the pypi
poetry/pdm + pip/uv workspace cases). That red is the intended "CI fails
if broken" signal — no underlying setup logic was changed here.
patches-api.socket.dev)and can go red on upstream 503s rather than a true regression; the hermetic
e2e-dockermatrix remains the always-on counterpart.required status checks (branch-protection UI — not configurable from the
workflow file).