Skip to content

feat: runner-aware tools#346

Draft
branchseer wants to merge 6 commits into
mainfrom
runner-aware-tools
Draft

feat: runner-aware tools#346
branchseer wants to merge 6 commits into
mainfrom
runner-aware-tools

Conversation

@branchseer
Copy link
Copy Markdown
Member

@branchseer branchseer commented Apr 18, 2026

Set up a IPC channel between vite-task and the processes it spawns, so the spawned tools can declare at runtime what they actually read, wrote, or cared about, and then vite-task uses that to decide what to fingerprint in the cache.

Design notes: docs/runner-task-ipc/.

Problems this PR solves

Every example below is exercised by patches/vite.patch, which wires vite build into the IPC through @voidzero-dev/vite-task-client.

1. Dynamic tracked envs

Before: the user had to declare every relevant env in vite-task.json, statically:

{
  "tasks": {
    "build": { "env": ["NODE_ENV", "VITE_*"], "cache": true }
  }
}

This duplicates knowledge the tool already has. Forgetting NODE_ENV silently skips cache invalidation on mode change. envPrefix-matching envs (VITE_* by default) get inlined into the bundle through import.meta.env.* — so changing envPrefix: 'MYAPP_' in vite.config.js without updating vite-task.json drifts: the runner still tracks VITE_* while the build output is driven by MYAPP_*.

After: the tool declares its envs at runtime, driven by its own config.

// vite's resolveConfig
fetchEnv("NODE_ENV", { tracked: true });

// vite's loadEnv, one call per configured prefix — these envs are
// exposed to client code as import.meta.env.*, so their values are
// baked into the bundle
for (const prefix of envPrefix) {
  fetchEnvs(`${prefix}*`, { tracked: true });
}

The build task in vite-task.json needs no env: at all. Changing envPrefix in vite.config.js dynamically changes the set of envs the runner tracks, with zero config edits on the runner side.

2. Exclude tool's cache dir from input/output

Vite stores pre-bundled deps under node_modules/.vite/ and bundled configs under node_modules/.vite-temp/. Every build reads the cache metadata (to check staleness) and writes fresh entries when it isn't stale. Without intervention the runner sees:

  • the reads → implicit inputs, so the cache key depends on dep-cache contents
  • the writes → implicit outputs
  • the same directory both read and written → the runner refuses to cache the run at all (read-write overlap)

There is a workaround already in vite-plus: voidzero-dev/vite-plus#1096 plus its follow-up #1198 hardcode !node_modules/.vite-temp/**, !node_modules/.vite/**/results.json, and !dist/** as negative input globs on every vp subcommand (build, test, pack). That's not good enough:

  • Leaks vite internals into vp. Every time vite changes its cache layout (new path under .vite/, moved temp dir, new subcommand with its own transient files), vp has to ship a matching glob update. It's a lockstep coupling that design-wise shouldn't exist.
  • Input-only, not symmetric. The globs suppress reads for the input fingerprint (which is enough to break the read-write overlap check), but the writes are still captured as outputs — meaning transient cache contents get archived into the runner's cache and restored on every hit, bloating the cache store.
  • Per-subcommand, per-tool duplication. #1198 already had to retrofit the same glob into three subcommands. Any new subcommand, and any third-party tool with similar behavior (Nuxt's .nuxt/, SvelteKit's .svelte-kit/, Next's .next/), needs its own hand-maintained list — vp can't ship it generically.

After:

// in loadCachedDepOptimizationMetadata
const depsCacheDir = getDepsCacheDir(environment);
ignoreInput(depsCacheDir);
ignoreOutput(depsCacheDir);

The declaration lives with the tool that owns the directory. The dep cache is vite's private concern.

3. Exclude output from input when a tool clears the folder before writing it

vite build calls emptyDir(outDir) before writing dist/. emptyDir has to read the directory entries to know what to delete — those reads look identical to genuine input reads. Since dist/ is also where vite writes its final output, the runner sees a read-write overlap on the same paths and refuses to cache.

After:

// in prepareOutDir, right before emptyDir()
ignoreInput(outDir);

Only the writes count. The pattern generalizes: any tool that wipes-then-writes the same directory needs to tell the runner "my enumeration reads aren't inputs."

What's in this PR

  • Step 1 — Protocol (vite_task_ipc_shared): message types + serialization shared by both ends.
  • Step 2 — Transport (vite_task_server + vite_task_client): async server, sync blocking client, tested Rust-to-Rust.
  • Step 3 — Extract artifact crate out of fspy for dylib embedding. (Landed on main via refactor: extract materialized_artifact crate out of fspy #344 as materialized_artifact.)
  • Step 4 — JS bridge: vite_task_client_napi + @voidzero-dev/vite-task-client JS wrapper (fetchEnv single-name + fetchEnvs glob, with dedupe against already-set process.env).
  • Step 5 — Runner integration: server started per task execution, client dylib embedded/extracted, IPC envs injected via serve()'s returned iterator.
  • Step 6 — Cache integration: runner consumes reported ignored inputs/outputs, tracked env requests (single + glob), and disable-cache signals when fingerprinting.

Test plan

  • Rust integration tests for server/client transport (vite_task_server/tests/integration.rs)
  • E2E snapshot fixtures per client method: ignore_input, ignore_output, fetch_env, fetch_envs_glob, disable_cache
  • E2E test caching a real vite build via patches/vite.patch (vite_build_cache fixture): NODE_ENV-change invalidation, envPrefix-driven tracked-env set change, dist/ write restoration on cache hit

@branchseer branchseer changed the base branch from main to graphite-base/346 April 20, 2026 02:19
@branchseer branchseer changed the base branch from graphite-base/346 to feat/output-restoration April 20, 2026 02:19
Copy link
Copy Markdown
Member Author

branchseer commented Apr 20, 2026

This stack of pull requests is managed by Graphite. Learn more about stacking.

@branchseer branchseer changed the title feat(ipc): runner-aware tools — protocol + transport (partial) feat: runner-aware tools Apr 20, 2026
@branchseer branchseer force-pushed the feat/output-restoration branch from 0008bd7 to 994624a Compare April 20, 2026 04:20
Comment thread packages/vite-task-client/index.js Outdated
@socket-security
Copy link
Copy Markdown

socket-security Bot commented Apr 23, 2026

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedcargo/​napi@​3.8.68210093100100
Addedcargo/​napi-build@​2.3.19810093100100
Addedcargo/​napi-derive@​3.5.59910093100100
Addedcargo/​ctor@​0.11.19810093100100
Addedcargo/​interprocess@​2.4.299100100100100

View full report

@branchseer branchseer force-pushed the runner-aware-tools branch 2 times, most recently from e18c21a to 09a310f Compare April 23, 2026 07:42
@branchseer branchseer force-pushed the feat/output-restoration branch 2 times, most recently from 589a626 to 9df3054 Compare April 23, 2026 08:00
@branchseer branchseer force-pushed the feat/output-restoration branch from 9df3054 to 3a8b605 Compare May 7, 2026 08:08
@branchseer branchseer force-pushed the runner-aware-tools branch from 4edcd12 to cfa4282 Compare May 7, 2026 08:08
@branchseer branchseer force-pushed the feat/output-restoration branch from 3a8b605 to 6198f6b Compare May 7, 2026 08:23
@branchseer branchseer force-pushed the runner-aware-tools branch 3 times, most recently from 24f5889 to ffca388 Compare May 7, 2026 08:38
branchseer and others added 2 commits May 14, 2026 18:32
Adds `{ auto: true }` support to the `output` field, plus the implicit
default: when `output` is omitted, automatically tracks files the task
writes (via fspy) and archives them. Explicit globs and `auto` can be
mixed in the same array.

Also includes:
- `read_write_overlap` check: if a task writes to a file it also read
  (auto-inferred), the cache update is skipped (`InputModified`).
  Prerun input hashes would otherwise be stale.
- Input negatives apply to reads only, not writes — keeps `input: ["!dist/**"]`
  from accidentally dropping writes to `dist/**` during archiving.
- Input-auto gating: when `input_config.includes_auto` is false, fspy
  reads do not contribute to the post-run fingerprint, even when fspy
  is enabled solely for output tracking.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Squashed rebase of 51 commits from runner-aware-tools branch onto
feat/output-restoration. Original commit history preserved on the
ras-backup ref for reference.

Key features bundled:
- vite_task_ipc_shared: shared protocol (Request/GetEnvResponse, NativeStr)
- vite_task_server: per-task IPC server (Handler trait + Recorder)
- vite_task_client: sync Rust client
- vite_task_client_napi + @voidzero-dev/vite-task-client: node addon + JS wrapper
- vite_task: wire IPC server into spawn; inject VP_IPC + VP_RUN_NODE_CLIENT_PATH;
  bundle with fspy via Tracking struct; materialize .node addon on first use
- consume runner-aware tool reports for cache decisions:
  * disableCache() short-circuits via ToolRequested
  * ignoreInput / ignoreOutput filter fspy reads/writes
  * tracked: true env / env-glob records folded into PostRunFingerprint
- IPC server failure surfaces via IpcServerError; cache update is skipped
- schema bumped to user_version = 13 (CacheEntryKey carries output_config,
  CacheEntryValue carries output_archive + tracked envs)

Conflicts resolved against post-rebase main (#352 cfg(fspy) gating, #321
output archiving, input-negative reads-only filter): TrackingOutcome
post-run summary preserved alongside Tracking pre-run handle; auto-input
reads gated on input_config.includes_auto and filtered by input negatives
+ ignoreInput; auto-output writes filtered by negatives + ignoreOutput.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@branchseer branchseer changed the base branch from feat/output-restoration to graphite-base/346 May 14, 2026 10:52
@branchseer branchseer force-pushed the graphite-base/346 branch from 6198f6b to c63db22 Compare May 14, 2026 10:52
@branchseer branchseer force-pushed the runner-aware-tools branch from ffca388 to 53d7bd1 Compare May 14, 2026 10:52
@branchseer branchseer changed the base branch from graphite-base/346 to main May 14, 2026 10:52
@branchseer
Copy link
Copy Markdown
Member Author

@codex review

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 53d7bd1883

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/vite-task-client/index.js Outdated
Comment thread crates/vite_task/src/session/execute/mod.rs Outdated
- Rename the `@voidzero-dev/vite-task-client` exports `fetchEnv`/`fetchEnvs`
  to `getEnv`/`getEnvs`: the calls are synchronous, so the `fetch*` naming
  was misleading (per review).
- `getEnv` now always consults the runner, even when `process.env[name]`
  is already set, so the dependency is still recorded for cache
  invalidation when the value was injected by the shell/prefix env.
- `collect_tracked_env_globs` no longer drops names already covered by the
  user's declared `env`. Lookup-time validation re-expands the glob over
  the whole parent env, so a filtered match-set always diffed as having
  `added` entries and missed the cache deterministically for tasks that
  both declare `env` and call `getEnvs` on an overlapping pattern.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@branchseer branchseer force-pushed the runner-aware-tools branch from ef023c1 to 7ba7e0a Compare May 15, 2026 02:40
branchseer and others added 3 commits May 15, 2026 11:11
`index.js` becomes plain JS (no JSDoc type annotations), and the type
declarations move to a new `index.d.ts` referenced from `package.json`'s
`types` field. Consumers like `vite` — whose strict tsconfig has
`isolatedDeclarations: true` and so can't enable `allowJs` — pick up
types from the `.d.ts` directly, without a separate generation step or
ambient module declaration.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Documents the vite-task ↔ vitejs/vite PR relationship, why the
vite-task-client-dist branch exists (pnpm subdir `&path:` doesn't
round-trip through `pnpm install`), and where the investigation can
resume — full test matrix, ruled-out knobs, and remaining angles.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
The runner-aware integration now lives upstream in vitejs/vite PR #22453,
so vite-task no longer needs to patch `vite` locally to wire the calls
into `@voidzero-dev/vite-task-client`. Both the root and playground
workspaces switch their `vite` catalog entry to the pkg.pr.new build of
the upstream PR, removing `patches/vite.patch`, the
`packageExtensions.vite` injection, and the dist-branch investigation
doc that's no longer relevant.

Until @voidzero-dev/vite-task-client is published to npm and the
integration ships in a real vite release, the `vite` catalog tracks
https://pkg.pr.new/vite@22453.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants