feat(e2e): AGX1-331 — event route authz black-box suite#277
Conversation
Set up the conventions for future resources before scope grows: - test-direct-resources: resources with their own SpiceDB type (currently api_key; agent and task slot in here later). - test-sub-resources: resources that delegate authz to a parent (currently event; state, message, tracker, checkpoint slot in here). - test-<resource>: all cases for one resource. - test-<resource>-<case>: one case for one resource. Aggregate groupings (DIRECT_RESOURCE_TESTS, SUB_RESOURCE_TESTS) are declared at the top so future PRs add a single line per resource rather than churning the target list. Added `make help` listing the convention. README's Run section rewritten to match. This commit lays the groundwork; AGX1-331 (PR #277) will pick up the new shape on rebase — its `test-events` target becomes `test-event` aligned with the singular-resource naming convention.
14d1221 to
430b1e8
Compare
Per Greptile review on #277: the test's previous name + docstring claimed to verify "BOTH query params are gated" by passing zero-UUIDs for both task_id and agent_id, but FastAPI evaluates the ``task_id`` Depends first and short-circuits before the agent_id gate runs. The test would still pass if the agent_id gate were removed entirely — it doesn't isolate either gate. Rename and rewrite the docstring to be honest about what it actually verifies: end-to-end route gating, no per-gate isolation. Independent isolation would require granting one resource but not the other in SpiceDB, which depends on a reachable spark-authz (out of scope today per Gap 1 in #276 — pubsec-dev runs raw SpiceDB without spark-authz).
Per Greptile review: ``make test-sub-resources`` and ``make test-event`` referenced ``tests/test_event_authz.py``, which only exists on #277. Running either target on this branch in isolation produced ``ERROR: not found: tests/test_event_authz.py`` and a non-zero exit. Remove ``EVENT_TESTS``, ``SUB_RESOURCE_TESTS``, ``test-sub-resources``, ``test-event``, plus their references in ``.PHONY``, the ``help`` block, and the README ``Run`` section. The Makefile's header comment retains the convention so #277 (and follow-up sub-resource tickets) can re-add the targets when their test files actually land.
Per Greptile review on #277: the test's previous name + docstring claimed to verify "BOTH query params are gated" by passing zero-UUIDs for both task_id and agent_id, but FastAPI evaluates the ``task_id`` Depends first and short-circuits before the agent_id gate runs. The test would still pass if the agent_id gate were removed entirely — it doesn't isolate either gate. Rename and rewrite the docstring to be honest about what it actually verifies: end-to-end route gating, no per-gate isolation. Independent isolation would require granting one resource but not the other in SpiceDB, which depends on a reachable spark-authz (out of scope today per Gap 1 in #276 — pubsec-dev runs raw SpiceDB without spark-authz).
…y exists #276's Greptile-driven cleanup removed test-event / test-sub-resources from the Makefile to avoid dangling targets that referenced a file which doesn't exist on that PR's branch. This PR adds the test file, so it now re-adds the matching targets + README references. After this commit (#277 in isolation): make test-direct-resources # api_key tests make test-sub-resources # event tests make test-api-key # all AGX1-325 cases make test-event # all AGX1-331 cases
f4c0961 to
2782f35
Compare
Per Greptile review on #277: the test's previous name + docstring claimed to verify "BOTH query params are gated" by passing zero-UUIDs for both task_id and agent_id, but FastAPI evaluates the ``task_id`` Depends first and short-circuits before the agent_id gate runs. The test would still pass if the agent_id gate were removed entirely — it doesn't isolate either gate. Rename and rewrite the docstring to be honest about what it actually verifies: end-to-end route gating, no per-gate isolation. Independent isolation would require granting one resource but not the other in SpiceDB, which depends on a reachable spark-authz (out of scope today per Gap 1 in #276 — pubsec-dev runs raw SpiceDB without spark-authz).
…y exists #276's Greptile-driven cleanup removed test-event / test-sub-resources from the Makefile to avoid dangling targets that referenced a file which doesn't exist on that PR's branch. This PR adds the test file, so it now re-adds the matching targets + README references. After this commit (#277 in isolation): make test-direct-resources # api_key tests make test-sub-resources # event tests make test-api-key # all AGX1-325 cases make test-event # all AGX1-331 cases
2782f35 to
fd3f448
Compare
Per Greptile review on #277: the test's previous name + docstring claimed to verify "BOTH query params are gated" by passing zero-UUIDs for both task_id and agent_id, but FastAPI evaluates the ``task_id`` Depends first and short-circuits before the agent_id gate runs. The test would still pass if the agent_id gate were removed entirely — it doesn't isolate either gate. Rename and rewrite the docstring to be honest about what it actually verifies: end-to-end route gating, no per-gate isolation. Independent isolation would require granting one resource but not the other in SpiceDB, which depends on a reachable spark-authz (out of scope today per Gap 1 in #276 — pubsec-dev runs raw SpiceDB without spark-authz).
fd3f448 to
e4fdf0c
Compare
…y exists #276's Greptile-driven cleanup removed test-event / test-sub-resources from the Makefile to avoid dangling targets that referenced a file which doesn't exist on that PR's branch. This PR adds the test file, so it now re-adds the matching targets + README references. After this commit (#277 in isolation): make test-direct-resources # api_key tests make test-sub-resources # event tests make test-api-key # all AGX1-325 cases make test-event # all AGX1-331 cases
Per Greptile review on #277: the test's previous name + docstring claimed to verify "BOTH query params are gated" by passing zero-UUIDs for both task_id and agent_id, but FastAPI evaluates the ``task_id`` Depends first and short-circuits before the agent_id gate runs. The test would still pass if the agent_id gate were removed entirely — it doesn't isolate either gate. Rename and rewrite the docstring to be honest about what it actually verifies: end-to-end route gating, no per-gate isolation. Independent isolation would require granting one resource but not the other in SpiceDB, which depends on a reachable spark-authz (out of scope today per Gap 1 in #276 — pubsec-dev runs raw SpiceDB without spark-authz).
…y exists #276's Greptile-driven cleanup removed test-event / test-sub-resources from the Makefile to avoid dangling targets that referenced a file which doesn't exist on that PR's branch. This PR adds the test file, so it now re-adds the matching targets + README references. After this commit (#277 in isolation): make test-direct-resources # api_key tests make test-sub-resources # event tests make test-api-key # all AGX1-325 cases make test-event # all AGX1-331 cases
e4fdf0c to
c6ab6da
Compare
Establishes the spark-authz e2e test harness in
``scripts/spark-authz-e2e-tests/`` and ships the first resource:
events (AGX1-331). Subsequent PRs (api_keys / AGX1-325, future
resources) layer test files on top of this infrastructure.
Models the EGP suite shape from scaleapi#142983 — same clients,
conftest pattern, cleanup-tracker, config.json layout.
What's in this PR
-----------------
- clients/agentex_client.py: HTTP client for scale-agentex routes
(agents + api_keys + events). api_key methods land here ahead of
AGX1-325 so the client doesn't grow in two places.
- clients/spark_authz_client.py: direct SpiceDB-state client (HTTP-
transcoded). Repo-agnostic; copied verbatim from the EGP suite.
- conftest.py: config loader, identity credentials, two AgentexClient
instances (user_a / user_b), SparkAuthzClient with graceful-skip
semantics (authz_reachable probes /healthz, returns True only on
2xx), ``parent_agent`` fixture that falls back to a pre-existing
``agentex.agent_id`` when the test user lacks ``agent.create``,
function-scoped cleanup tracker.
- helpers/{cleanup,factories}.py: LIFO teardown + unique-name gens.
- Makefile / README: four-terminal local-run recipe + per-resource
+ logical-group targets.
- tests/test_event_authz.py: AGX1-331 event-route tests.
AGX1-331 scope: event get/list delegated to parent agent view
(read-only); no agent view -> 404 on get and filtered/empty list.
Tests (test_event_authz.py)
---------------------------
- test_get_event_nonexistent_returns_404 — sanity: ItemDoesNotExist
short-circuits before any authz fires.
- test_list_events_without_agent_view_returns_404 — user_b lacks
read on user_a's agent -> DAuthorizedQuery collapses to 404.
- test_list_events_denied_on_both_query_params_returns_404 — route
is gated end-to-end (does not isolate which gate fires; isolation
needs SpiceDB-side grants).
- test_get_event_with_view_returns_200 — SKIPPED. No public
POST /events; would need an event-seeding helper that doesn't
belong in an authz-scoped e2e PR.
Verified against pubsec-dev
---------------------------
3 passed / 1 skipped / 0 failed against
``agentex.sgp-pubsec-dev.scale.com`` on 2026-06-04 with
``spark_authz.host`` port-forwarded to the in-cluster spark-authz
(``kubectl -n spark port-forward svc/spark-authz 8090:8090``).
Linear: AGX1-331. Companion follow-up: AGX1-325 (api_keys) lands its
test files on top of this scaffolding once pubsec-dev's agentex-auth
is redeployed with AUTH_PROVIDER=spark (deployment-config blocker
tracked separately in scaleapi/sgp#3334).
c6ab6da to
cfa002d
Compare
|
Can we just reuse https://github.com/scaleapi/scale-agentex/tree/3a75e52a042e8aa9928fab66867f43ff6c71771b/agentex/tests/integration. I guess the main difference is that these changes hit REST api but integration tests hit |
| return f"{scheme}://{self.host}" | ||
|
|
||
|
|
||
| class SparkAuthzClient: |
There was a problem hiding this comment.
Spark AuthZ is a downstream microservice, we shouldnt be surfacing the client to be hit directly here. It should be tested in isolation downstream. Here we just want to test agentex functionality (if that in turns calls spark thats fine)
|
Closing — moving this suite to the private Replacement: scaleapi/agentex#365 — same scaffolding + AGX1-331 event tests, with the reviewer's spark-authz-boundary fix already applied (no The AGX1-325 api_keys follow-up is scaleapi/agentex#366, stacked on #365. |
Summary
Establishes the spark-authz e2e test harness in
scripts/spark-authz-e2e-tests/and ships the first resource: events (AGX1-331). Subsequent PRs (api_keys / AGX1-325 — see #276 — and future resources) layer test files on top of this infrastructure.Models the EGP suite shape from scaleapi#142983 — same clients, conftest pattern, cleanup-tracker,
config.jsonlayout.Linear: AGX1-331.
What this PR ships
Scaffolding (reused by all future resource test files):
clients/agentex_client.py— HTTP client forscale-agentexroutes (agents + api_keys + events). api_key methods land here ahead of AGX1-325 so the client doesn't grow in two places.clients/spark_authz_client.py— direct SpiceDB-state client (HTTP-transcoded). Repo-agnostic; copied verbatim from the EGP suite.conftest.py— config loader, identity credentials, twoAgentexClientinstances (user_a / user_b),SparkAuthzClientwith graceful-skip semantics (authz_reachableprobes/healthz, returnsTrueonly on 2xx),parent_agentfixture that falls back to a pre-existingagentex.agent_idwhen the test user lacksagent.create, function-scoped cleanup tracker.helpers/{cleanup,factories}.py— LIFO teardown + unique-name generators.Makefile/README.md— four-terminal local-run recipe + per-resource + logical-group targets.AGX1-331 tests (
tests/test_event_authz.py):test_get_event_nonexistent_returns_404ItemDoesNotExistshort-circuits before any authz firestest_list_events_without_agent_view_returns_404user_blacksreadonuser_a's agent →DAuthorizedQueryonagent_idcollapses denial to 404test_list_events_denied_on_both_query_params_returns_404test_get_event_with_view_returns_200POST /events; would need an event-seeding helper out of scope for an authz-focused PRVerified against pubsec-dev
Ran against
agentex.sgp-pubsec-dev.scale.comon 2026-06-04:Setup used:
config.jsonwith a real pubsec-dev API key + the EGPaccount_idasx-selected-account-id.agentex.agent_idset to a pre-existing agent (the user lacksagent.createon the tenant wildcard, soparent_agentfalls back to the configured id).spark_authz.hostpointed atlocalhost:8090viakubectl port-forwardtons/spark/svc/spark-authz. The event tests don't assert against SpiceDB directly, so they pass without it too; the port-forward is only required when api_key SpiceDB-asserting tests get added on top (see feat(e2e): AGX1-325 — agent_api_keys authz black-box suite #276).Out of scope
Greptile Summary
Introduces a black-box FGAC test suite for the
eventsroutes (GET /events/{id}andGET /events), along with all the scaffolding — httpx clients, conftest fixtures, LIFO cleanup tracker, and a Makefile with per-resource and aggregate targets — needed to host future sub-resource test files.POST /eventsto seed test data), leaving a clear hook for whoever adds an event-seeding helper later.parent_agentfixture provides a fallback to a pre-existingagentex.agent_idfrom config for environments where the test user lacksagent.create, and theauthz_clientfixture degrades gracefully to a skip when spark-authz isn't reachable.Confidence Score: 5/5
This PR adds only a new test script directory with no changes to production application code — it is safe to merge.
All changes are confined to a standalone e2e test suite under
scripts/. The denied-path assertions are straightforward and were verified against pubsec-dev. No production code, migrations, or auth logic is touched.No files require special attention.
Important Files Changed
Sequence Diagram
sequenceDiagram participant TB as Test (user_b) participant AX as scale-agentex participant AA as agentex-auth participant SDB as SpiceDB (via spark-authz) TB->>AX: "GET /events?agent_id=X&task_id=Y" AX->>AA: resolve principal (forward headers) AA-->>AX: "{user_id, account_id}" AX->>SDB: DAuthorizedQuery: check agent:X read for user_b SDB-->>AX: DENIED (no tuple) AX-->>TB: 404 (existence not leaked) Note over TB,SDB: GET /events/{nonexistent_id} short-circuits before authz TB->>AX: GET /events/00000000-… AX->>AX: repo lookup → ItemDoesNotExist AX-->>TB: 404 (no authz call)Prompt To Fix All With AI
Reviews (7): Last reviewed commit: "feat(e2e): AGX1-331 — event authz black-..." | Re-trigger Greptile
Context used:
Learned From
scaleapi/scaleapi#126926
Learned From
scaleapi/scaleapi#127117