Skip to content

[CI] Add weekly conformance run against latest upstream release#304

Open
soyuka wants to merge 1 commit into
modelcontextprotocol:mainfrom
soyuka:ci-weekly-conformance
Open

[CI] Add weekly conformance run against latest upstream release#304
soyuka wants to merge 1 commit into
modelcontextprotocol:mainfrom
soyuka:ci-weekly-conformance

Conversation

@soyuka
Copy link
Copy Markdown
Contributor

@soyuka soyuka commented May 12, 2026

The PR pipeline only runs conformance when someone opens a PR. Upstream @modelcontextprotocol/conformance releases between PRs go untested.

This workflow:

  • runs the same conformance commands on a weekly cron (Mondays 06:00 UTC),
  • can be triggered manually from the Actions tab (workflow_dispatch),
  • pins to @modelcontextprotocol/conformance@latest to pick up new releases,
  • opens (or comments on) a tracking issue labelled conformance-weekly on scheduled failures.

Test plan

  • Manual run from the Actions tab succeeds (or fails cleanly with an issue filed)

The PR pipeline only catches conformance changes when someone opens a
PR. Upstream `@modelcontextprotocol/conformance` releases between PRs
go untested. This adds a weekly cron + `workflow_dispatch` trigger
pinned to `@latest`, and opens a tracking issue on scheduled failures.
uses: actions/github-script@v7
with:
script: |
const marker = '<!-- conformance-weekly-failure -->';
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure we really need this notify section, what do you think about running this weekly?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm in favor for giving it a go - if we don't act on it and things just pile up, we can kill it easily again

Comment thread .github/workflows/conformance-weekly.yaml
@soyuka
Copy link
Copy Markdown
Contributor Author

soyuka commented May 15, 2026

global opinion about this? should I clean it up?

@chr-hertel
Copy link
Copy Markdown
Member

+1 from my end, yes :)
was wondering about creating an issue, but it's the best fit for our tools

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants