Skip to content

fix(cli): remove eval export option#1349

Merged
christso merged 1 commit into
mainfrom
cleanup/remove-eval-export
Jun 10, 2026
Merged

fix(cli): remove eval export option#1349
christso merged 1 commit into
mainfrom
cleanup/remove-eval-export

Conversation

@christso

Copy link
Copy Markdown
Collaborator

Summary

agentv eval now has one results boundary: the --output run directory and its index.jsonl manifest. The removed --export path no longer appears in help or parsing, and former flat-format writers are gone so JSONL/benchmark/timing/transcript/per-test artifacts remain the only eval-run artifact contract.

Consumer research

  • Repo-local search found --export only in AgentV docs, tests, helper scripts, and the eval CI workflow; all were migrated to --output / index.jsonl or removed.
  • GitHub code search for "agentv eval" "--export", "agentv eval run" "--export", and WiseTechGlobal/EntityProcess scoped variants returned no external hits.
  • gh repo view WiseTechGlobal/sdd confirmed the repo is private; downloaded the accessible zipball and searched it directly. The only agentv eval usages were .github/workflows/sdd-eval.yml and evals/e2e/scripts/check-v2-suite.test.cjs, and neither used --export.

Behavior changes

  • agentv eval run --help no longer advertises --export.
  • Passing --export is rejected by the CLI parser as an unknown argument.
  • Helper scripts that previously copied flat result files now read or copy <run>/index.jsonl from the canonical run folder.
  • The CI eval workflow now uploads the run folder and summary without trying to publish JUnit XML from agentv eval.

Red/green CLI evidence

Scenario Before After
Help output agentv eval run --help listed --export <str> no --export option in help
--export invocation dry-run accepted --export /tmp/.../results.jsonl and wrote the flat file exits 1 with Unknown arguments
Canonical output --output /tmp/.../run wrote index.jsonl, benchmark.json, timing.json, transcript.jsonl unchanged; the same artifacts are written under the run directory

--output remains the canonical complete artifact boundary with index.jsonl, benchmark.json, timing.json, transcript.jsonl, per-test artifacts, and task bundles.

Verification

  • bun run verify
  • bun run validate:examples
  • bun test apps/cli/test/eval.integration.test.ts
  • Green UAT: bun apps/cli/src/cli.ts eval ... --dry-run --output /tmp/.../run --export /tmp/.../results.jsonl fails as unknown; --output /tmp/.../run writes the expected run artifacts.

Code review: manual simplification/review completed; subagent-based Tier 2 review was not run because this Codex session's multi-agent tool only permits spawning when the user explicitly asks for sub-agents. The simplification pass removed the now-dead flat-format writer dispatch and tests.

Post-Deploy Monitoring & Validation

No additional operational monitoring required. This removes an unreleased/cleanup CLI option; validation is via CI and user-facing CLI behavior (agentv eval run --help, agentv eval ... --output <dir>, and rejection of --export).


Compound Engineering
Codex

@christso christso force-pushed the cleanup/remove-eval-export branch from 537d08f to 5b690bf Compare June 10, 2026 12:10
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 10, 2026

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 5b690bf
Status: ✅  Deploy successful!
Preview URL: https://d47941ea.agentv.pages.dev
Branch Preview URL: https://cleanup-remove-eval-export.agentv.pages.dev

View logs

@christso christso merged commit 80a4389 into main Jun 10, 2026
8 checks passed
@christso christso deleted the cleanup/remove-eval-export branch June 10, 2026 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant