Skip to content

Add experimental pull request artifact tools - #3843

Open
talum wants to merge 3 commits into
mainfrom
pr-artifacts-support
Open

talum wants to merge 3 commits into
mainfrom
pr-artifacts-support

Conversation

@talum

@talum talum commented Nov 13, 2026 •

Copy link
Copy Markdown

Summary

Add experimental pull request artifact tools for listing, upserting, and deleting artifacts, following the issue-artifact tool pattern. Tool exposure requires the explicit MCP feature flag pull_request_artifacts; GitHub separately enforces pull_request_artifacts_api for the target repository.

Why

Enable clients to read and write PR artifacts through MCP without treating a user-level tool-exposure flag as repository rollout authorization.

What changed

  • Add pull_request_artifact_read (method: list) and pull_request_artifact_write (method: upsert | delete) in the pull_requests toolset, with exported typed inputs and outputs.
  • Resolve repository IDs and use the PR artifact API's name-based mutation routes. Preserve commit selection and expose next_page from REST Link headers.
  • Add the explicit pull_request_artifacts MCP flag, off by default and excluded from Insiders. Each API request remains subject to GitHub's repository/organization-aware pull_request_artifacts_api gate.
  • Add input/response validation, sanitization, IFC labeling, lockdown rejection, schema snapshots, regression tests, feature documentation, and reasoned skills-coverage exemptions.

MCP impact

  • No tool or API changes
  • Tool schema or behavior changed
  • New tool added

Two new experimental tools; existing issue-artifact tools are unchanged. Read-only mode exposes only the read tool. Clients opt in with --features=pull_request_artifacts, GITHUB_FEATURES=pull_request_artifacts, or the supported HTTP feature configuration.

Prompts tested (tool changes only)

  • Natural-language client prompts were tested through Copilot CLI using the locally built github-artifacts-local server. The prompt sequence is recorded below. Manual mcpcurl schema discovery was also confirmed after exporting GITHUB_PERSONAL_ACCESS_TOKEN; the write tool advertises --method, --name, and --body.
  • Manual prompt testing used the scratch artifact named mcp-test on github/pull-request-review-testing#1885. Enabled-versus-disabled repository rollout toggling was not exercised.

Manual prompt sequence

To repeat the manual prompt testing, connect github-artifacts-local using the setup below and use these prompts in order. Create/update/delete modify the scratch artifact; substitute a different test PR or artifact name if needed.

Create

Use github-artifacts-local to upsert an artifact named mcp-test on github/pull-request-review-testing#1885 with body "Testing artifacts through natural language."

Read

Use github-artifacts-local to list artifacts across all commits on github/pull-request-review-testing#1885, including their bodies.

Update

Use github-artifacts-local to update the artifact named mcp-test on github/pull-request-review-testing#1885 to body "Updated through natural language." Use upsert.

Delete

Use github-artifacts-local to delete the artifact named mcp-test from github/pull-request-review-testing#1885.

Verify deletion

Use github-artifacts-local to list artifacts across all commits on github/pull-request-review-testing#1885 and confirm mcp-test is absent.

Security / limits

  • No security or limits impact
  • Auth / permissions considered
  • Data exposure, filtering, or token/size limits considered

Both tools declare the existing repo OAuth scope policy. GitHub's API enforces operation-specific authorization and the per-repository rollout flag; exact fine-grained permission mappings were not independently verified. All operations reject lockdown mode before making network requests.

Artifact names and creator logins use plain-text sanitization; bodies use the existing content sanitizer. List returns one page, defaulting to 30 and capped at 100. Writes are not automatically retried; conflicts, permission errors, throttling, and storage failures are returned explicitly. A 404 is not interpreted as definitive evidence of a disabled rollout flag.

Tool renaming

  • I am renaming tools as part of this PR (e.g. a part of a consolidation effort)
    • I have added the new tool aliases in deprecated_tool_aliases.go
  • I am not renaming tools as part of this PR

Note: if you're renaming tools, you must add the tool aliases. For more information on how to do so, please refer to the official docs.

No existing tools are renamed.

Lint & tests

  • Linted locally with ./script/lint
  • Tested locally with ./script/test

Completed verification:

Command Result
go test ./pkg/github ./pkg/skills/catalog -run '^(TestPullRequestArtifact|TestIssueArtifact|TestResolveFeatureFlags|TestCatalogToolRequirementsAreRegistered|TestCatalogAvailabilityModes)' -count=1 Passed
UPDATE_TOOLSNAPS=true go test ./... Passed; intentional tool snapshots generated
script/lint Passed, 0 issues
script/test Passed, full go test -race ./... suite
script/generate-docs Passed; generated changes are limited to docs/feature-flags.md
git diff --check Passed

The required lint, race-test, and documentation commands passed in that order. Regression coverage includes create/update upsert statuses, delete, commit selection, Link-based pagination, escaped paths, invalid inputs, legacy/modern output, sanitization/IFC labels, lockdown, API errors/no retries, and mocked enabled/disabled repository responses under the same request context. The automated live E2E suite was not run, and repository rollout flags were not changed.

Reproducing locally

Use Go 1.27.3 or later. Run the following from the repository root, keeping authentication and invocation commands in the same terminal.

Build and automated tests

git switch pr-artifacts-support

mkdir -p bin
go build -o bin/github-mcp-server ./cmd/github-mcp-server
go build -o bin/mcpcurl ./cmd/mcpcurl

# Focused tests use mocks and do not require a GitHub token.
go test ./pkg/github -run '^TestPullRequestArtifact' -count=1

# Full validation after changes.
script/lint && script/test && script/generate-docs

Authentication and discovery

The target repository must have the API rollout flag pull_request_artifacts_api enabled, and the token must have access to the target PR. The current MCP scope policy requires repo for classic PATs. The separate MCP exposure flag is pull_request_artifacts; passing it does not enable the repository's API rollout.

# Alternatively, export a suitable PAT from your normal credential source.
export GITHUB_PERSONAL_ACCESS_TOKEN="$(gh auth token)"

SERVER="./bin/github-mcp-server stdio --features=pull_request_artifacts --toolsets=pull_requests"

./bin/mcpcurl --stdio-server-cmd "$SERVER" schema

./bin/mcpcurl --stdio-server-cmd "$SERVER" \
  tools pull_request_artifact_write --help

Discovery should advertise both artifact tools, and write-tool help should include --method. If mcpcurl reports unknown flag: --method, run schema directly: startup/discovery failures can otherwise surface as misleading unknown-flag errors. In particular, having a PAT configured in gh is not sufficient unless the server receives GITHUB_PERSONAL_ACCESS_TOKEN. A 404 from an artifact operation may indicate disabled repository rollout, missing resources, or insufficient access.

Direct MCP operation tests

These commands mutate only the named scratch artifact mcp-test on github/pull-request-review-testing#1885. Substitute a repository/PR you are authorized to test.

# Create.
./bin/mcpcurl --stdio-server-cmd "$SERVER" \
  tools pull_request_artifact_write \
  --method upsert \
  --owner github --repo pull-request-review-testing --pull_number 1885 \
  --name mcp-test --body "Temporary test artifact"

# Read across all commits.
./bin/mcpcurl --stdio-server-cmd "$SERVER" \
  tools pull_request_artifact_read \
  --method list \
  --owner github --repo pull-request-review-testing --pull_number 1885 \
  --commit_oid all

# Update the same artifact by name.
./bin/mcpcurl --stdio-server-cmd "$SERVER" \
  tools pull_request_artifact_write \
  --method upsert \
  --owner github --repo pull-request-review-testing --pull_number 1885 \
  --name mcp-test --body "Updated test artifact"

# Delete.
./bin/mcpcurl --stdio-server-cmd "$SERVER" \
  tools pull_request_artifact_write \
  --method delete \
  --owner github --repo pull-request-review-testing --pull_number 1885 \
  --name mcp-test

# Verify deletion; follow next_page if the result is paginated.
./bin/mcpcurl --stdio-server-cmd "$SERVER" \
  tools pull_request_artifact_read \
  --method list \
  --owner github --repo pull-request-review-testing --pull_number 1885 \
  --commit_oid all

Expected results: create/update return an artifact; list returns a list with bodies and optional next_page; delete returns a deleted confirmation; the final list should no longer include the scratch artifact. Omit commit_oid or use current to test current-head selection; use a fixed commit OID for stable selection across pages.

Connect the local build to Copilot CLI

Configure the token explicitly on the MCP server rather than relying on environment inheritance. Run this from the repository root in the shell where the token was exported:

# Only when replacing an entry created during an earlier test:
# copilot mcp remove github-artifacts-local

copilot mcp add github-artifacts-local \
  --env "GITHUB_PERSONAL_ACCESS_TOKEN=${GITHUB_PERSONAL_ACCESS_TOKEN:?Export your PAT first}" \
  --tools pull_request_artifact_read,pull_request_artifact_write \
  -- "$(pwd)/bin/github-mcp-server" stdio \
  --features=pull_request_artifacts \
  --toolsets=pull_requests

--env saves the token in the local Copilot configuration (~/.copilot/mcp-config.json, or under COPILOT_HOME when overridden). Do not share or commit that file or include token values in test reports.

Start copilot, or in an existing session reload and inspect the server:

/mcp reload
/mcp list
/mcp get github-artifacts-local

Use the manual prompt sequence above and confirm the tool calls target github-artifacts-local. After future code changes, rebuild the server binary and run /mcp reload before retesting; the existing process does not pick up a newly built binary automatically.

Docs

  • Not needed
  • Updated (README / docs / examples)

Updated docs/feature-flags.md with experimental opt-in behavior, the distinction between the two feature flags, operations, pagination, output, and safety constraints.

Skills

  • Updated relevant skills or confirmed no bundled skill applies

PR artifact management is outside the current bundled workflows. Added explicit exemptions for both tools in pkg/skills/catalog/availability_test.go, following the issue-artifact precedent; catalog coverage checks passed.

Expose list, upsert, and delete operations behind the pull_request_artifacts MCP feature flag, with repository rollout enforced by the GitHub API. Include commit-aware pagination, lockdown handling, tool snapshots, regression tests, and documentation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7bc67675-583d-4ef1-9b2a-25ae850acee2
Comment thread pkg/github/pull_request_artifacts.go Dismissed
Comment thread pkg/github/pull_request_artifacts.go Resolved
@talum talum self-assigned this Nov 13, 2026
@talum
talum requested a dynamic review from Copilot November 13, 2026 17:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The artifact name transformation loses addressing information, so passing a listed name to upsert or delete can overwrite or delete a different artifact.

1 open issue
What changed in this PR

Adds experimental pull request artifact management to the GitHub MCP Server’s pull_requests toolset, following the existing issue-artifact pattern.

Changes:

  • Introduces typed tools for listing artifacts and upserting or deleting them by name, with commit selection and pagination.
  • Requires explicit feature opt-in, separate from GitHub’s repository rollout gate, and supports read-only and lockdown restrictions.
  • Adds validation, sanitization, security labels, regression tests, schema snapshots, and documentation.
File Description
pkg/​skills/​catalog/​availability_test.go Adds reasoned skill-coverage exemptions for both tools.
pkg/​github/​tools.go Registers the artifact tools.
pkg/​github/​pull_request_artifacts.go Implements tool schemas, API operations, validation, and result handling.
pkg/​github/​pull_request_artifacts_test.go Covers discovery, requests, pagination, errors, and safety behavior.
pkg/​github/​pull_request_artifact_types.go Defines exported artifact input and output types.
pkg/​github/​feature_flags.go Adds the explicit artifact-tool exposure flag.
pkg/​github/​__toolsnaps__/​pull_request_artifact_write_ff_pull_request_artifacts.snap Records the write tool’s schema.
pkg/​github/​__toolsnaps__/​pull_request_artifact_read_ff_pull_request_artifacts.snap Records the read tool’s schema.
docs/​feature-flags.md Documents opt-in setup, operations, outputs, and constraints.

Review effort: Max (dynamic)

Comment thread pkg/github/pull_request_artifacts.go Resolved
Add target-bound, expiring MCP confirmation for pull request artifact deletion and lossless name keys for safe read-to-write round trips.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7bc67675-583d-4ef1-9b2a-25ae850acee2
Comment thread pkg/github/pull_request_artifacts.go Resolved
Comment thread pkg/github/pull_request_artifacts.go Resolved
@talum
talum requested review from BigDadBear and dstello November 13, 2026 19:37
@talum
talum marked this pull request as ready for review November 13, 2026 19:37
@talum
talum requested a review from a team as a code owner November 13, 2026 19:37

@BigDadBear BigDadBear left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Zero experience with this repo but lgtm!

func pullRequestArtifactResult(ctx context.Context, deps ToolDependencies, repository *gogithub.Repository, output any) *mcp.CallToolResult {
result := MarshalledTextResult(output)
if shouldAttachIFCLabel(ctx, deps, result) && repository.Private != nil {
setIFCLabel(result, ifc.LabelRepoUserContent(*repository.Private))

@Lukeghenco Lukeghenco left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants