Action posts repo's CLAUDE.md verbatim as comment body instead of structured review (workflow exits success)

[IAmAlexander]

Summary

anthropics/claude-code-action@v1 intermittently posts the repository's CLAUDE.md file verbatim as its PR comment, instead of producing the structured review the workflow's prompt: requests. The action exits conclusion: success, so the workflow check is green, but no real review happened.

Environment

• Action: anthropics/claude-code-action@v1 (floating tag — version drift may be relevant)
• Trigger: pull_request (synchronize event on a human-pushed commit)
• Auth: claude_code_oauth_token
• Model: action-selected default

Repro

Same workflow ran on two consecutive commits of the same PR. First run = normal structured review. Second run = CLAUDE.md echo.

Run ID	Conclusion	Comment shape
25720844923 (initial)	success	676-char structured security review, ends with the configured verdict-tag HTML comment
25722050413 (1-line YAML follow-up)	success	19,407-char raw CLAUDE.md verbatim, no verdict tag

First 120 chars of the misfire comment body:

```

CLAUDE.md

Guidance for Claude Code (claude.ai/code) working in this repository.

User context lives in USER.md (working
```

This is the exact opening of our repo's CLAUDE.md file (top-level guidance file — see https://docs.claude.com/en/docs/claude-code/memory).

Workflow config (security-review.yml)

```yaml

• uses: anthropics/claude-code-action@v1
  with:
  claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
  github_token: ${{ secrets.GITHUB_TOKEN }}
  show_full_output: true
  prompt: |
  Perform a security review of this pull request. ...
  (template requests structured output with severity classification + verdict tag)
  claude_args: '--allowed-tools "Bash(gh pr view:),Bash(gh pr diff:),Bash(gh pr comment:*)"'
  ```

Note: claude_args only allows gh pr view/diff/comment. The model cannot directly cat CLAUDE.md under these tool restrictions — so the file's content must be reaching the model via the action's own internal context-loading step, not via a model-issued tool call.

Hypothesis

CLAUDE.md is auto-loaded into the model's context (per --memory semantics of Claude Code, which the action wraps). Intermittently, the model conflates content loaded as guidance with content to post, and pipes CLAUDE.md verbatim into gh pr comment. The structured review template is silently skipped.

The 1-line YAML-comment-only diff between the two commits suggests this is not content-triggered by the PR diff itself — it's a flake in the action's prompt-wrapper layer.

Expected

Either:

1. The action produces the structured review the prompt: template requested, OR
2. The action fails the run with conclusion: failure if the model output is non-conforming.

It should not exit success with an unrelated dump.

Impact

• Self-detecting setups (workflows with verdict-tag fail-safes that parse the final line for a binding token) correctly skip auto-approve — no bad approvals ship.
• Naive setups (workflows that treat green check = review happened) silently accept a non-review.
• Wastes workflow minutes + OAuth quota when retried.

Suggested fixes

1. The action could detect "model output is suspiciously similar to a memory file it loaded" (≥10kb body whose head matches a loaded guidance file) and retry-or-fail rather than post.
2. The action could separate the model's "guidance context" (read-only memory) from "documents available for tool calls" (Bash/Read targets), so guidance is never re-emittable as content.
3. Strengthen the prompt-template scaffolding around the review-output schema (e.g. require a tagged opening line) so non-conforming output is rejected at the action layer, not by the consumer's parser.

Workaround

Re-trigger via workflow_dispatch (or via a @claude review please comment on workflows that use the separate claude.yml pattern). Both yielded the correct structured output on retry — supports the "flake, not deterministic" interpretation.

Happy to provide additional run logs or correlate further with action-side telemetry if useful.