GitHub Action

Lint prompts in CI with one composite Action

Drop riccardomerenda/promptscore@main into any GitHub Actions workflow to score every prompt in your repo on every push.

Quick start

The Action is a composite that wraps the public CLI. It installs Node, runs npx @promptscore/cli analyze against the paths you give it, and exits non-zero when findings cross the configured severity threshold.

name: PromptScore

on:
  pull_request:
  push:
    branches: [main]

jobs:
  prompt-lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: riccardomerenda/promptscore@main
        with:
          inputs: prompts/
          model: claude
          format: markdown
          fail-on: warning

When format: markdown is set, the Action also appends the report to the job summary, so reviewers see findings inline in the GitHub UI without opening the workflow log.

Inputs

InputDefaultDescription
inputsprompts/Files, directories, or globs to analyze. Whitespace-separated for multiple paths.
model_baseProfile name. Built-in: _base, claude, gpt. Custom profiles supported via project config.
formattextOutput format: text, markdown, or json.
fail-onerrorSeverity threshold that fails the action: error, warning, info, or none.
config—Path to a PromptScore config file. Empty falls back to project discovery.
rules—Comma-separated rule IDs to include. Empty means all rules from the active profile.
include-llmfalseRun LLM-backed rules. Requires the relevant API key in the runner environment.
cli-versionlatestVersion of @promptscore/cli to install. Pin to a specific version for reproducibility.
node-version20Node.js version to use on the runner.

Common patterns

Block PRs that introduce ambiguous prompts

- uses: riccardomerenda/promptscore@main
  with:
    inputs: prompts/
    model: claude
    fail-on: warning

Failing on warning catches everything from missing tasks to vague qualifiers. Drop to fail-on: error if you only want to gate on the highest-impact issue (currently missing-task).

Run LLM-backed review on changed prompts only

- name: Find changed prompts
  id: changed
  run: |
    git fetch origin main
    files=$(git diff --name-only origin/main...HEAD -- 'prompts/*.txt' 'prompts/*.md' \
      | tr '\n' ' ')
    echo "files=$files" >> "$GITHUB_OUTPUT"

- if: steps.changed.outputs.files != ''
  uses: riccardomerenda/promptscore@main
  with:
    inputs: ${{ steps.changed.outputs.files }}
    model: claude
    include-llm: true
    fail-on: warning
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

The opt-in llm-prompt-review rule is more expensive and not needed on every push. Run it only on changed files in PR builds, with the API key available via secrets.

Pin to a specific CLI version for reproducibility

- uses: riccardomerenda/promptscore@main
  with:
    inputs: prompts/
    cli-version: '0.4.7'

The cli-version input pins the underlying npm package version, so reruns produce identical results even if a newer release ships in the meantime.

Versioning

The Action is versioned alongside the rest of PromptScore. Reference riccardomerenda/promptscore@main to track the latest tagged release, or pin to a specific tag (e.g. @v0.4.8) for full reproducibility. The action.yml contract is part of the package’s public surface, so breaking changes will land on a major version bump.

What it does not do yet

  • It does not annotate individual lines in PR diffs (no inline review comments).
  • It does not cache npm downloads across runs. Each run does a fresh npx -y; pinning cli-version still works but does not save the install.
  • It does not produce a SARIF report yet, so it does not show up in GitHub’s Security tab.

These are reasonable next steps once the Action sees real usage. Open an issue if any of them blocks you.

PreviousBrowser AnalyzerNextRules Reference