# Automate with Claude Code Source: https://getquality.md/claude-code Run recurring evaluations and hand off recommendations on a cadence with Claude Code. Recurring evaluation keeps a QUALITY.md model honest: the same `/quality evaluate` you run interactively can run on a schedule, with the evaluation artifacts committed back to the repository. This guide shows the agent-first setup in Claude Code. For the model behind the cadence, see [Engineering loops](/loops). ## How evaluation runs inside Claude Code When Claude Code runs `/quality evaluate`, the skill selects the built-in `harness` evaluator by default (unless you or the workspace config choose another evaluator). The deterministic `qualitymd` runner owns the work graph and every artifact; the Claude Code session itself inspects requirement-specific workspace context and supplies judgment plus evidence locators through checkpoints. That means: * **No provider API key is needed.** Evaluation judgment uses the same authenticated agent that is already running the skill. * **No nested agent process.** The runner never launches a second Claude or Codex session; it exchanges typed JSON requests and results with the current one. * **Interrupted runs resume.** Every checkpoint is persisted in the run's `evaluation.json`, so a stopped session picks up the pending work request with `qualitymd evaluation run --resume --json`. ## Choose an execution surface * **Cloud routines** run as autonomous sessions against a fresh clone of your repository on a schedule, API call, or GitHub trigger. See the [Claude Code routines documentation](https://code.claude.com/docs/en/routines) and the [cloud environment reference](https://code.claude.com/docs/en/claude-code-on-the-web). * **Local scheduled tasks and session loops** run on your machine inside an ordinary Claude Code session. They use your local checkout and stay available only while the app and machine are. ## Set up a recurring evaluation routine 1. **Make the skill and CLI available in the environment.** Commit the `/quality` project skill to the repository so fresh clones carry it, and install a compatible `qualitymd` in the environment's setup script — for example through the install script or npm package from the [Quickstart](/quickstart). 2. **Invoke the skill explicitly in the routine prompt.** A reliable prompt names the skill and the scope rather than describing the task loosely: ```text theme={null} Run /quality evaluate for the repository's QUALITY.md. When the run completes, commit the new evaluation run folder and open a pull request summarizing the rating, top findings, and top recommendations. ``` 3. **Decide how artifacts persist.** Cloud routines work on a fresh clone, so anything not committed or pushed is gone when the session ends. Have the routine commit the new `.quality/evaluations//` folder (the authoritative `evaluation.json`, generated reports, and logs) to a branch or pull request. Local tasks write directly to your working tree. 4. **Grant the permissions evaluation needs.** The run writes only under the resolved evaluation directory (default `.quality/evaluations/`) and the workflow feedback log under `.quality/logs/`. Network access is not required for harness-backed judgment; it may be needed while the environment installs `qualitymd` or authenticates an explicitly selected agent runtime. ## Unattended behavior The evaluate workflow adds no interactive gates when run unattended: the run advances checkpoint by checkpoint, finishes with a report, or stops with the runner's classified failure and remedy. An `awaiting_evaluator` receipt is normal progress — if a session ends mid-run, the next session resumes the run and recovers the same pending work request. `qualitymd status --json` and `qualitymd evaluation list --json` identify awaiting runs and their continuation command. ## Agent-runtime authentication Harness-backed evaluation needs no credential beyond the routine's own agent. If you explicitly select the `claude` or `codex` evaluator, authentication belongs to that coding-agent runtime and may use its documented login, subscription, or headless credential mechanism. `qualitymd` does not define API-key evaluator methods or store provider tokens in `.quality/config.yaml`. Supply any runtime credential through the environment's secret facility and avoid exposing job-wide credentials to repository-controlled code. # CLI reference Source: https://getquality.md/cli The qualitymd command-line interface that the /quality skill builds on. The `/quality` skill drives the `qualitymd` CLI for you. This reference is for when you want to run commands directly. It is generated from the CLI's own command definitions, so it always matches the installed binary. ## Commands | Command | Description | | ---------------------- | ------------------------------------------------------------- | | `qualitymd evaluation` | Work with QUALITY.md evaluation runs | | `qualitymd init` | Scaffold a starter QUALITY.md | | `qualitymd lint` | Validate a QUALITY.md file | | `qualitymd model` | Query a quality model's structure and canonical reference IDs | | `qualitymd spec` | Emit the QUALITY.md format specification | | `qualitymd schema` | Emit the companion JSON Schema for QUALITY.md frontmatter | | `qualitymd status` | Show a QUALITY.md workspace status snapshot | | `qualitymd update` | Update the qualitymd CLI through its owning install channel | | `qualitymd version` | Show structured qualitymd version metadata | ## `qualitymd evaluation` Work with QUALITY.md evaluation runs ```bash theme={null} qualitymd evaluation <subcommand> [flags] ``` **Subcommands** | Command | Description | | ----------------------------- | --------------------------------------------------------------- | | `qualitymd evaluation run` | Execute a complete evaluation run with the deterministic runner | | `qualitymd evaluation create` | Create a numbered evaluation run folder | | `qualitymd evaluation list` | List evaluation runs | | `qualitymd evaluation status` | Show whether an evaluation run is reportable | | `qualitymd evaluation data` | Work with evaluation structured data | | `qualitymd evaluation report` | Build evaluation reports | ### `qualitymd evaluation run` Execute a complete evaluation run with the deterministic runner ```bash theme={null} qualitymd evaluation run [flags] ``` **Flags** | Flag | Description | | -------------------- | ----------------------------------------------------------------------------------------------- | | `--model` | QUALITY.md file to evaluate | | `--evaluation-dir` | Override the model-relative evaluation directory | | `--area` | Canonical area reference for the evaluation scope | | `--factor` | Canonical factor reference for a scoped evaluation; repeatable | | `--evaluator` | Evaluator to use: auto (default), harness, codex, claude, or a configured agent-runtime profile | | `--resume` | Resume an existing run from its evaluation.json | | `--evaluator-result` | Submit harness result envelopes for outstanding work requests | | `--dry-run`, `-n` | Preview the resolved run without invoking an evaluator or writing evaluation data | | `--json` | Emit a machine-readable run receipt | ### `qualitymd evaluation create` Create a numbered evaluation run folder ```bash theme={null} qualitymd evaluation create [flags] [<model>] ``` **Flags** | Flag | Description | | ------------------ | -------------------------------------------------------------- | | `--model` | QUALITY.md file to snapshot | | `--area` | Canonical area reference for the evaluation scope | | `--factor` | Canonical factor reference for a scoped evaluation; repeatable | | `--evaluation-dir` | Override the model-relative evaluation directory | | `--json` | Emit a machine-readable run creation receipt | ### `qualitymd evaluation list` List evaluation runs ```bash theme={null} qualitymd evaluation list [flags] ``` **Flags** | Flag | Description | | ------------------ | -------------------------------------------------- | | `--model` | QUALITY.md file that anchors evaluation history | | `--evaluation-dir` | Override the model-relative evaluation directory | | `--state` | Filter runs: all, reportable, incomplete, awaiting | | `--json` | Emit a machine-readable run list | ### `qualitymd evaluation status` Show whether an evaluation run is reportable ```bash theme={null} qualitymd evaluation status [flags] [<run>] ``` **Flags** | Flag | Description | | ------------------ | -------------------------------------------------------------------- | | `--latest` | Use the most recent evaluation run | | `--evaluation-dir` | Override the model-relative evaluation directory when using --latest | | `--model` | QUALITY.md file that anchors model-relative run paths | | `--json` | Emit a machine-readable status document | ### `qualitymd evaluation data` Work with evaluation structured data ```bash theme={null} qualitymd evaluation data <subcommand> [flags] ``` **Subcommands** | Command | Description | | ----------------------------------- | -------------------------------------------------------- | | `qualitymd evaluation data set` | Validate and persist a batch of evaluation JSON payloads | | `qualitymd evaluation data list` | List stored evaluation JSON payloads | | `qualitymd evaluation data get` | Print one stored evaluation JSON payload | | `qualitymd evaluation data kinds` | List evaluation data kinds | | `qualitymd evaluation data example` | Print a complete evaluation example JSON payload | | `qualitymd evaluation data schema` | Print the evaluation structured data JSON Schema | | `qualitymd evaluation data verify` | Validate persisted evaluation JSON payloads | #### `qualitymd evaluation data set` Validate and persist a batch of evaluation JSON payloads ```bash theme={null} qualitymd evaluation data set [flags] [<run>] ``` **Flags** | Flag | Description | | ------------------ | -------------------------------------------------------------------- | | `--latest` | Use the most recent evaluation run | | `--evaluation-dir` | Override the model-relative evaluation directory when using --latest | | `--model` | QUALITY.md file that anchors model-relative run paths | | `--dry-run`, `-n` | Validate and report intended write without persisting | | `--json` | Emit a machine-readable write receipt | #### `qualitymd evaluation data list` List stored evaluation JSON payloads ```bash theme={null} qualitymd evaluation data list [flags] [<run>] ``` **Flags** | Flag | Description | | ------------------ | -------------------------------------------------------------------- | | `--latest` | Use the most recent evaluation run | | `--evaluation-dir` | Override the model-relative evaluation directory when using --latest | | `--model` | QUALITY.md file that anchors model-relative run paths | | `--kind` | Filter by evaluation data kind | | `--json` | Emit a machine-readable data list | #### `qualitymd evaluation data get` Print one stored evaluation JSON payload ```bash theme={null} qualitymd evaluation data get [flags] [<run>] ``` **Flags** | Flag | Description | | ------------------ | -------------------------------------------------------------------- | | `--latest` | Use the most recent evaluation run | | `--evaluation-dir` | Override the model-relative evaluation directory when using --latest | | `--model` | QUALITY.md file that anchors model-relative run paths | | `--kind` | Evaluation data kind | | `--area` | Area ref | | `--factor` | Factor ref | | `--requirement` | Requirement ref | | `--selector` | Optional sub-result selector | | `--json` | Not supported: data get already emits JSON | #### `qualitymd evaluation data kinds` List evaluation data kinds ```bash theme={null} qualitymd evaluation data kinds [flags] ``` **Flags** | Flag | Description | | -------- | -------------------------------------- | | `--json` | Emit a machine-readable data kind list | #### `qualitymd evaluation data example` Print a complete evaluation example JSON payload ```bash theme={null} qualitymd evaluation data example [flags] <kind> ``` **Flags** | Flag | Description | | -------- | ---------------------------------------------- | | `--json` | Not supported: data example already emits JSON | #### `qualitymd evaluation data schema` Print the evaluation structured data JSON Schema ```bash theme={null} qualitymd evaluation data schema [flags] [<kind>] ``` **Flags** | Flag | Description | | -------- | --------------------------------------------- | | `--json` | Not supported: data schema already emits JSON | #### `qualitymd evaluation data verify` Validate persisted evaluation JSON payloads ```bash theme={null} qualitymd evaluation data verify [flags] [<run>] ``` **Flags** | Flag | Description | | ------------------ | -------------------------------------------------------------------- | | `--latest` | Use the most recent evaluation run | | `--evaluation-dir` | Override the model-relative evaluation directory when using --latest | | `--model` | QUALITY.md file that anchors model-relative run paths | | `--json` | Emit a machine-readable verification receipt | ### `qualitymd evaluation report` Build evaluation reports ```bash theme={null} qualitymd evaluation report <subcommand> [flags] ``` **Subcommands** | Command | Description | | ----------------------------------- | ------------------------------------------------------------- | | `qualitymd evaluation report build` | Build the complete Markdown report tree for an evaluation run | #### `qualitymd evaluation report build` Build the complete Markdown report tree for an evaluation run ```bash theme={null} qualitymd evaluation report build [flags] [<run>] ``` **Flags** | Flag | Description | | ------------------ | -------------------------------------------------------------------- | | `--latest` | Use the most recent evaluation run | | `--evaluation-dir` | Override the model-relative evaluation directory when using --latest | | `--model` | QUALITY.md file that anchors model-relative run paths | | `--json` | Emit a machine-readable report build receipt | ## `qualitymd init` Scaffold a starter QUALITY.md ```bash theme={null} qualitymd init [flags] [<path>] ``` **Flags** | Flag | Description | | ------------------------- | ---------------------------------------------------------------- | | `--force` | Overwrite an existing file | | `--json` | Emit a machine-readable JSON init receipt | | `--minimal` | Write a minimal valid skeleton without the guided template prose | | `--no-agent-instructions` | Do not create or update agent instruction files | ## `qualitymd lint` Validate a QUALITY.md file ```bash theme={null} qualitymd lint [flags] [<path>] ``` **Flags** | Flag | Description | | -------- | ---------------------------------------- | | `--json` | Emit a machine-readable JSON lint result | | `--fix` | Apply deterministic in-place repairs | ## `qualitymd model` Query a quality model's structure and canonical reference IDs ```bash theme={null} qualitymd model <subcommand> [flags] ``` **Subcommands** | Command | Description | | ---------------------- | ------------------------------------------------- | | `qualitymd model tree` | Render the model as a containment hierarchy | | `qualitymd model list` | Enumerate model elements with their canonical IDs | | `qualitymd model get` | Show one element's detail and immediate relations | ### `qualitymd model tree` Render the model as a containment hierarchy ```bash theme={null} qualitymd model tree [flags] [<path>] ``` **Flags** | Flag | Description | | --------- | ---------------------------------------------------------- | | `--json` | Emit the tree as nested JSON | | `--area` | Root the tree at a canonical area reference (area:\) | | `--depth` | Limit nesting depth; 0 emits only the rooted node | ### `qualitymd model list` Enumerate model elements with their canonical IDs ```bash theme={null} qualitymd model list [flags] [<path>] ``` **Flags** | Flag | Description | | -------- | ---------------------------------------------------- | | `--json` | Emit the enumeration as a JSON array | | `--area` | Restrict output to one area's subtree (area:\) | | `--type` | Restrict output to kinds: area, factor, requirement | ### `qualitymd model get` Show one element's detail and immediate relations ```bash theme={null} qualitymd model get [flags] <id [path]...> ``` **Flags** | Flag | Description | | -------- | ---------------------------------------- | | `--json` | Emit the element detail as a JSON object | ## `qualitymd spec` Emit the QUALITY.md format specification ```bash theme={null} qualitymd spec [flags] <argument...> ``` ## `qualitymd schema` Emit the companion JSON Schema for QUALITY.md frontmatter ```bash theme={null} qualitymd schema [flags] <argument...> ``` ## `qualitymd status` Show a QUALITY.md workspace status snapshot ```bash theme={null} qualitymd status [flags] [<path>] ``` **Flags** | Flag | Description | | -------- | ------------------------------------------------- | | `--json` | Emit a machine-readable workspace status snapshot | ## `qualitymd update` Update the qualitymd CLI through its owning install channel ```bash theme={null} qualitymd update [flags] ``` **Flags** | Flag | Description | | --------- | --------------------------------------------- | | `--check` | Check for a newer release without applying it | | `--json` | Emit a machine-readable update result | ## `qualitymd version` Show structured qualitymd version metadata ```bash theme={null} qualitymd version [flags] ``` **Flags** | Flag | Description | | -------- | -------------------------------------- | | `--json` | Emit machine-readable version metadata | # Automate with Codex Source: https://getquality.md/codex Run recurring evaluations and hand off recommendations on a cadence with Codex. Recurring evaluation keeps a QUALITY.md model honest: the same `$quality` evaluation you run interactively can run as a Codex scheduled task, with the evaluation artifacts persisted in your project. This guide shows the agent-first setup in Codex. For the model behind the cadence, see [Engineering loops](/loops). ## How evaluation runs inside Codex When Codex runs the quality skill's evaluate workflow, the skill selects the built-in `harness` evaluator by default (unless you or the workspace config choose another evaluator). The deterministic `qualitymd` runner owns the work graph and every artifact; the Codex session itself inspects requirement-specific workspace context and supplies judgment plus evidence locators through checkpoints. That means: * **No provider API key is needed.** Evaluation judgment uses the same authenticated agent that is already running the skill. * **No nested agent process.** The runner never launches a second Codex or Claude session; it exchanges typed JSON requests and results with the current one. * **Interrupted runs resume.** Every checkpoint is persisted in the run's `evaluation.json`, so a stopped task picks up the pending work request with `qualitymd evaluation run --resume --json`. ## Set up a recurring evaluation task 1. **Create the scheduled task** from ChatGPT on the web or the Codex desktop app, per the [Codex scheduled-task documentation](https://learn.chatgpt.com/docs/automations?surface=app), and select the local project or worktree the QUALITY.md model lives in. 2. **Make the CLI available.** Install a compatible `qualitymd` on the machine or in the task environment — see [Quickstart](/quickstart) — so the skill can drive the runner. 3. **Invoke the skill explicitly in the task prompt.** Repeatable workflows name the skill rather than describing it: ```text theme={null} Run the $quality evaluate workflow for this repository's QUALITY.md and summarize the rating, top findings, and top recommendations. ``` 4. **Keep workspace write access.** The run writes the numbered evaluation run folder under the resolved evaluation directory (default `.quality/evaluations/`) plus the workflow feedback log under `.quality/logs/`, so the task needs workspace-write access to persist its artifacts. Local tasks run against your checkout and require the machine and app to remain available; cloud environments distinguish task-wide environment variables from setup-only secrets, per the [Codex cloud environment reference](https://learn.chatgpt.com/docs/environments/cloud-environment). ## Unattended behavior The evaluate workflow adds no interactive gates when run unattended: the run advances checkpoint by checkpoint, finishes with a report, or stops with the runner's classified failure and remedy. An `awaiting_evaluator` receipt is normal progress — a later task or session resumes the run and recovers the same pending work request. `qualitymd status --json` and `qualitymd evaluation list --json` identify awaiting runs and their continuation command. ## Agent-runtime authentication Harness-backed evaluation needs no credential beyond the task's own agent. If you explicitly select the `codex` or `claude` evaluator, authentication belongs to that coding-agent runtime and may use its documented login, subscription, or headless credential mechanism. `qualitymd` does not define API-key evaluator methods or store provider tokens in `.quality/config.yaml`. Supply any runtime credential through the environment's secret facility and avoid exposing task-wide credentials to repository-controlled code. # Welcome Source: https://getquality.md/index Model your project's quality so teams and agents can evaluate it, stay aligned on what good means, and improve it over time.

QUALITY.md

Open format Agent skill CLI

Engineer loops to improve project quality, aligning your team and agents on what quality means for what matters most.

What is QUALITY.md?

QUALITY.md is an open file format for declaring a project quality model: a holistic rubric of the desired quality characteristics (security, maintainability, code quality, test/spec quality, etc.), their quality requirements (guides, sensors, principles, standards, rules, specs), and the context (mission/purpose, user/stakeholder needs, risks, etc.) for why they matter.

The QUALITY.md format specification is designed to be agent, tool, and vendor agnostic. The companion `/quality` agent skill and `qualitymd` CLI serve as both a reference implementation and starter kit for working with `QUALITY.md` files, including the ability to create a comprehensive quality evaluation report with prioritized improvement recommendations ready for handoff or further review, useful for engineering quality loops or augmenting your existing workflow.

Why QUALITY.md?

  • Ensure stakeholders are aligned on what matters most and why
  • Identify the most impactful and low risk quality improvements
  • Build agentic control loops to continuously maintain and improve quality
  • Manage quality proactively instead of reactively
  • Tackle intent, comprehension, and technical debt
  • Move judgement and taste up the loop stack
  • Compound learning at the highest leverage point

Philosophy

Maintaining the quality of a thing is caring for it. When a thing exists for the benefit of another, caring for it well is caring for the other well.

The aim of QUALITY.md is to help others care well by making the practice of maintenance accessible to humans and agents.

How to use QUALITY.md?

  1. Install the `/quality` agent skill:

    ```sh theme={null} npx skills add qualitymd/quality.md ```

  2. Install the `qualitymd` CLI:

    ```sh theme={null} npm install -g quality.md ```

    See getting started for other install methods.

  3. Invoke `/quality setup` to create a tailored `QUALITY.md` for evaluation and continued refinement.

Run `/quality evaluate` to generate a holistic quality report with findings and actionable recommendations. See the [example quality evaluation report](https://github.com/qualitymd/quality.md/blob/main/examples/report-gallery/software-service/.quality/evaluations/0001-full-eval/report.md). Hand off quality improvement recommendations to inner agent loops, teammates, or the workflow best suited to the change. Run `/quality improve` to refine `QUALITY.md` from what the evaluation and improvement work revealed. Return to Evaluate with a sharper model and continue compounding quality learning.
Create your first QUALITY.md to establish a project quality model. Elevate quality evaluation and improvement up the loop stack. See project quality ratings by project area and quality factor. The open, vendor-neutral, QUALITY.md standard. Effortlessly set up, evaluate, and improve. QUALITY.md linting and management utilities.

FAQ

A quality model is an explicit, shared description of what "good" means for something you maintain. In QUALITY.md, that means the areas of a project, the quality factors that matter for each, and concrete requirements with checkable assessments — plus a rating scale for judging results and the context for why any of it matters. Instead of quality living in people's heads and scattered review comments, the model makes it visible enough for teams and agents to evaluate, align on, and improve. The idea has a long lineage. Manufacturing quality management (Shewhart's statistical quality control, Juran's "fitness for use") carried into software through McCall's and Boehm's models in the 1970s, HP's FURPS, Dromey's model, and the ISO 9126 and ISO 25010 standards. Across five decades the same pattern recurs: decompose quality into the characteristics that matter for a particular thing and its stakeholders, refine them into sub-characteristics, and tie those to assessable criteria. In practice, these models rarely spread beyond large organizations with formal quality programs: building and tailoring a detailed model was costly, and the metric-based rating methods meant to make quality measurable proved too reductive — or too complex — for everyday use. AI changes that. Agents make a tailored quality model cheap to establish and maintain, and inferential judging of quality requirements makes holistic evaluation accessible — good enough for many use cases where formal measurement never was. The [specification](/specification) defines the file format and its meaning: document structure (YAML frontmatter plus a Markdown body), the frontmatter schema for areas, factors, requirements, and the rating scale, naming and reference rules, and model semantics (how sources resolve, how requirements scope to their declaring area, how factors connect, and what the rating scale means). Conformance is a property of documents; the spec doesn't prescribe or certify any evaluation method. It's deliberately agent-, tool-, and vendor-agnostic. The [`/quality` skill](/skill) and [`qualitymd` CLI](/cli) are companions — a reference implementation and starter kit, not part of the format itself. No. The format is domain agnostic: it applies wherever you want to be deliberate about quality — documentation sets, data products, research reports, service operations, and more. There's no default factor checklist to import; each model earns its factors from the modeled thing's own needs and risks. Yes. The specification, `/quality` agent skill, and `qualitymd` CLI are all developed in the open at [github.com/qualitymd/quality.md](https://github.com/qualitymd/quality.md) and released under the MIT license. They're complementary layers. Rules files like AGENTS.md or CLAUDE.md tell an agent *how* to work in your project, and skills give it procedures for specific tasks — both steer the inner coding loop, action by action. `QUALITY.md` declares *what quality means*: the shared bar the resulting work is judged against. The `/quality` skill connects the two — it reads your `QUALITY.md` to evaluate the project and produces recommendations you can hand off to any agent, teammate, or workflow. In loop terms, rules and skills guide the inner loop; QUALITY.md powers the evaluation and improvement loops above it.
# Engineering quality loops Source: https://getquality.md/loops Continuously evaluate and improve project quality >

> **Note:** The `/quality` agent skill and `qualitymd` CLI can create a > comprehensive quality evaluation with findings and actionable > recommendations that are ideal for engineering a quality improvement loop > based on how QUALITY.md models project quality. >

> >

> While not part of the formal spec, the skill and CLI demonstrate how a tool > or system can build on the QUALITY.md standard. Future models, agents, and > AI assistants can become more capable of using QUALITY.md to guide, > evaluate, and improve their work products. >

> >

> The rest of this guide demonstrates how the quality evaluation created by > the skill and CLI can be integrated into workflows or agentic control loops > to help maintain and improve project quality. >

## The quality loop ### Basic form 1. **Observe:** user and stakeholder needs, objectives, risks, internal and external feedback, conversations, changes to the environment, etc. 2. **Orient:** refine or correct QUALITY.md based on observations and an assessment of previous recommendation quality: credibility, utility, and risk; then evaluate quality with `/quality evaluate` to produce a comprehensive quality evaluation report with verifiable improvement recommendations for consideration. 3. **Decide:** determine recommendations that are ready for handoff or implementation. 4. **Act:** hand off or implement actionable recommendations. # Quickstart Source: https://getquality.md/quickstart Install the /quality skill and qualitymd CLI, then create your first QUALITY.md. This guide takes you from nothing to a first `QUALITY.md` tailored to your project. ## Install ### Install the agent skill ```sh theme={null} npx skills add qualitymd/quality.md ``` ### Install the CLI Choose one CLI install method. Use npm or pnpm if you already use Node.js; use the hosted installer when you want a managed standalone install. ```sh npm theme={null} npm install -g quality.md ``` ```sh pnpm theme={null} pnpm add -g quality.md ``` ```sh macOS/Linux theme={null} curl -fsSL https://getquality.md/install.sh | sh ``` ```powershell Windows PowerShell theme={null} iwr https://getquality.md/install.ps1 -UseB | iex ``` ```sh Homebrew theme={null} brew install qualitymd/tap/qualitymd ``` ## Create your first QUALITY.md Run `/quality setup` with your coding agent for guided creation of a `QUALITY.md` tailored to your project. ## Stay up to date Run `/quality update` to update the skill and CLI to the latest compatible versions. ## Next steps Evaluate, review, act, and improve. Document structure, model vocabulary, and model semantics. # /quality agent skill Source: https://getquality.md/skill The agent skill is the primary way to set up, evaluate, and improve a QUALITY.md. The `/quality` skill is the primary interface to QUALITY.md. You work through an AI assistant or coding agent; the skill manages the `QUALITY.md` file and the underlying CLI for you. ## Commands | Command | What it does | | ------------------- | ------------------------------------------------ | | `/quality setup` | Guided creation of an initial `QUALITY.md`. | | `/quality evaluate` | Produce a quality evaluation report. | | `/quality improve` | Refine `QUALITY.md` and compound learning. | | `/quality update` | Update the skill and CLI to the latest versions. | ## How the skill uses the CLI You still just run `/quality evaluate`: the skill frames the run, resolves the scope, and then drives `qualitymd evaluation run`, the CLI's deterministic evaluation runner, which produces the evaluation artifacts and reports. See the [CLI reference](/cli) for the deterministic commands the skill builds on. # Specification Source: https://getquality.md/specification The QUALITY.md format specification — document structure, model vocabulary, frontmatter schema, and model semantics. **Specification version:** 0.12 (Draft) This document specifies the QUALITY.md standard: a Markdown file with YAML frontmatter that declares a quality model and a Markdown body that documents its context. The specification is a reference for authors and for tools that read, write, or exchange QUALITY.md documents and need to interpret them consistently. This specification defines the document structure, model vocabulary, frontmatter schema, and model semantics of QUALITY.md: what a conforming document is, and what a conforming document means. It does not define how a model is to be evaluated. Evaluation methods, aggregation approaches, report formats, and tool behavior are outside its scope; [Appendix D](#appendix-d-an-illustrative-evaluation-method) illustrates one evaluation method without making it normative. Authoring advice, examples, and notes are informative unless explicitly stated otherwise. The specification version identifies the QUALITY.md document format and semantics defined here. See [Versioning](https://github.com/qualitymd/quality.md/blob/main/docs/reference/versioning.md#specification-version) for the project's specification-version policy. ## Lineage Note: This section is non-normative. QUALITY.md is domain agnostic: a model can describe quality for software, documents, data sets, research or analytical reports, services, operations, processes, or other evaluated entities. It is informed by established software-quality, requirements, measurement, testing, and evaluation traditions, including ISO/IEC and ISO/IEC/IEEE standards, CISQ structural-quality work, and earlier software quality models such as McCall and Dromey. These are acknowledged as influences, not normative references or conformance targets. QUALITY.md uses its own vocabulary and structure where that makes the format more practical, readable, and accessible. It takes useful boundaries and vocabulary from those traditions, not their characteristic lists as default factors. ## Conformance Conformance is a property of QUALITY.md documents. A QUALITY.md document conforms when it satisfies all applicable normative requirements in this specification. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14, [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119), and [RFC 8174](https://www.rfc-editor.org/rfc/rfc8174) when, and only when, they appear in all capitals, as shown here. All content in this specification is normative except sections or passages explicitly marked as non-normative, informative, examples, or notes. Examples and notes are non-normative. They illustrate intended interpretation, motivation, or common edge cases; they do not add conformance requirements. This specification defines no conformance classes for tools. Parsers, linters, evaluators, and report renderers are expected to interpret conforming documents according to the vocabulary and semantics defined here, but how a model is evaluated, aggregated, or reported is a method choice this specification does not constrain. An interpretation that contradicts [Model semantics](#model-semantics) is a misreading of the document; an unsound evaluation method applied to a correctly read document is outside this specification's scope. A document can carry additional frontmatter properties as described in [Extensions](#extensions). ## Terminology In this specification, bare "requirement" always means the model object defined below; conformance obligations of this specification are called "normative requirements" or "conformance requirements". **Quality model**: A structured, declarative description of what quality means for a root area and any child areas. **Entity**: A thing evaluated for quality. **Model**: The root object in a QUALITY.md file. A model is the root area plus the model-wide rating scale. **Area**: An entity or set of entities with quality requirements subject to evaluation. **Area name**: A single map key under `areas`, unique among sibling areas in that `areas` map. **Area ID**: The ordered path of area names from the root area to an area. The root area ID is the empty path. **Area title**: The required human display label stored in an area's `title`. **Source**: A selector describing the material evaluated by an area. **Factor**: A quality characteristic or attribute through which an area's quality is described. A factor groups connected requirements and can be decomposed into sub-factors. **Factor name**: A single map key under `factors`, unique among sibling factors in that `factors` map. **Factor ID**: The declaring area ID plus the ordered path of factor names from that area's `factors` map to the factor. **Requirement**: An assessable quality expectation. A requirement has a stable requirement name, a title, an assessment, zero or more explicit factor references, and optional per-level criterion overrides. **Assessment**: The means for assessing an area's source against a requirement, stated inline or as a reference to an entity that defines those means. **Rating scale**: The ordered set of rating levels used by a model. **Rating level**: A single level on a rating scale, with a stable meaning and a default criterion for judging a requirement at that level. **Rating level ID**: The `level` value of a rating level, unique within the model's rating scale. **Requirement name**: A requirement's stable map key, unique within its declaring area. **Model reference**: A text form used at human/tool boundaries to address an area, factor, requirement, or rating level. **Display value**: A human-facing label for a known model concept. Display values are not model references unless a section explicitly says so. ## Names and model references Area names, factor names, requirement names, and rating level IDs MUST match: ```regex theme={null} ^[A-Za-z0-9](?:[A-Za-z0-9_-]*[A-Za-z0-9])?$ ``` The grammar excludes `/`, `:`, spaces, dots, and leading or trailing separators so canonical model references are unambiguous and do not resemble filesystem paths. Area names MUST NOT be `root`. The token `root` is reserved for the root area in area, factor, and requirement references. Qualified area references use `area:`. The root area reference is `area:root`; nested area references join area names with `/`, for example `area:webhooks` and `area:webhooks/delivery`. Qualified factor references use `factor:::`. The root declaring area is written as `root`, for example `factor:root::security` and `factor:root::security/secrets`. Nested declaring areas and nested factors use `/` within each side of the `::` separator, for example `factor:webhooks/delivery::reliability/retry-behavior`. Qualified requirement references use `requirement:::`. The root declaring area is written as `root`, for example `requirement:root::release-notes-current` and `requirement:webhooks/delivery::retry-window`. Qualified rating level references use `rating:`, for example `rating:target`. A qualified model reference is valid only when it uses one of the typed prefixes `area:`, `factor:`, `requirement:`, or `rating:`, every segment matches the strict name grammar, and the referenced model element is declared in the model. A value that fails any of these conditions is not a qualified model reference. An unqualified reference omits the type prefix: unqualified area references render as `root` or ``, unqualified factor references render as `::`, and unqualified rating level references render as the rating level ID. An unqualified reference is meaningful only where the surrounding context fixes the reference type. On mixed-reference surfaces, and anywhere the reference type must be recoverable from the value alone — including durable machine-readable artifacts — only qualified references are unambiguous. Display values are human-facing labels, not references. The root area's display value is `/`; nested areas display as ``, such as `webhooks/delivery`. `/` is never a model reference, qualified or unqualified. Factor display values render as ``; rating level display values render as the rating level ID unless a title is resolved from the model. ## Document structure A QUALITY.md document is a Markdown file containing: 1. A YAML frontmatter block containing the model. 2. An optional Markdown body documenting the model's context. The document MUST begin with a valid YAML frontmatter block. The frontmatter MUST contain a conforming model. The Markdown body can be empty. The body is free-form Markdown: no headings, sections, or ordering are required, and none are reserved. Unrecognized headings or sections do not affect a document's conformance. The location of a QUALITY.md document defines the default source for the root area: the directory containing the file and all descendants. A root area can override that default by declaring `source`. ## Frontmatter schema Every property defined in this section MUST, when present, use the YAML shape specified here. Frontmatter whose defined properties do not conform to these shapes is not a conforming QUALITY.md document. Properties this section does not define are extension properties; see [Extensions](#extensions). Null or empty values do not satisfy required properties. A required property with a null or empty value MUST be treated as absent. > **Non-normative.** A companion JSON Schema for this frontmatter is available as > `quality.schema.json` (emitted by `qualitymd schema`). It is structural-only — > it describes the shapes in this section, not the semantic rules that follow > from this specification — and is subordinate to this specification and to its > durable spec (`specs/quality-schema-json.md`). Passing structural validation > does not imply document conformance. ### Model A model is the root node of a QUALITY.md document. It has all area properties plus the model-wide `ratingScale`. ```yaml theme={null} title: # Required description: # Optional ratingScale: # Required - level: # Required; unique within the scale title: # Required description: # Recommended criterion: # Required factors: # Optional* : requirements: # Optional* : areas: # Optional* : source: # Optional ``` An entry on either `factors`, `requirements`, or `areas` MUST be supplied. `ratingScale` is unique to the model. An area MUST NOT declare `ratingScale`. ### Rating scale `ratingScale` MUST be a sequence of at least two rating levels ordered from best to worst. Each rating level MUST declare: * `level`: a non-empty scalar rating level ID unique within the rating scale. * `title`: a non-empty scalar human-readable label. * `criterion`: a non-empty scalar default criterion for judging whether a requirement sits at that rating level. Each rating level SHOULD declare: * `description`: the stable meaning of the level across the model. The meaning of `description`, `criterion`, scale order, and criterion overrides is defined in [Rating scale semantics](#rating-scale-semantics). ### Area An area is the recursive node of the model. Each entry under `areas` is an area. ```yaml theme={null} title: # Required description: # Optional factors: # Optional* : requirements: # Optional* : areas: # Optional* : source: # Optional ``` An area can declare no `factors` or `requirements` of its own when it is used as a grouping node for child `areas`. `title` is the area's display name. The area's map key remains its area name; its area ID is the ordered path of area names from the root area to that area. When present, `source` is a single string selecting the material evaluated by the area. A selector is not limited to filesystem material: relative paths and globs resolve relative to the containing QUALITY.md file, and any other selector describes the evaluated subject in prose — a saved query, a live system, a body of records ("open tickets in the support queue") — interpreted by the evaluating method. The selector identifies what is judged; it does not precompute all context an evaluator may need. See [Source semantics](#source-semantics). Source inheritance and the scope of an area's requirements are defined in [Model semantics](#model-semantics). ### Factor A factor groups requirements through a quality characteristic. ```yaml theme={null} title: # Required description: # Recommended factors: # Optional : requirements: # Optional : ``` `factors`, when present on a factor, declares sub-factors. A sub-factor is a factor of the same shape, nested to any depth. Factor identity is local to the area on which the factor is declared. Factors with the same name on different areas are distinct factors. Within one area, authors SHOULD avoid reusing the same factor name anywhere in that area's recursive factor tree. Canonical factor IDs remain path-based, but explicit requirement factor references use scalar names, so repeated names in one area can make authored references ambiguous to readers and tools. `title` is the factor's display name. The factor's map key remains its stable factor name local to the area where the factor is declared. Its factor ID is the declaring area ID plus the ordered path of factor names from that area's `factors` map to the factor. ### Requirement A requirement is identified by its map key, the stable requirement name. A requirement MUST declare a `title` and exactly one `assessment`. ```yaml theme={null} title: # Required description: # Optional assessment: # Required factors: # Optional; required for direct area requirements - ratings: # Optional : ``` The requirement name MUST match the strict name grammar and MUST be unique within the declaring area. For uniqueness, requirements declared directly under an area and requirements declared under that area's factors or sub-factors are considered to belong to the declaring area. `title` is the requirement's human-facing statement. The requirement's map key is its stable requirement name. `assessment` MUST be a single non-empty scalar. A missing, empty, null, or list-valued `assessment` is invalid. An `assessment` either states the means of assessing inline or references an entity that defines them, such as a specification, guide, or checklist. Referencing names that entity once instead of copying criteria that would drift from their origin. Note: This note is non-normative. A referenced entity may itself be an area in the model. Referencing it by the same selector used as that area's `source` makes the dependency traceable from the requirement to that area without a distinct link type. Every requirement MUST be connected to at least one factor. A requirement declared under a factor or sub-factor is connected by placement. Such a requirement can also declare `factors`; those entries are same-area secondary factor references. A requirement declared directly under an area is not connected by placement. It MUST declare `factors` with at least one non-empty scalar entry. Each explicit factor reference MUST resolve to a factor declared on the area where the requirement sits. A factor declared on an ancestor area, sibling area, descendant area, or unrelated area is not in scope for that requirement's `factors` entries. Missing `factors`, `factors: null`, `factors: []`, and sequences containing only null or empty entries do not connect a direct area requirement to any factor, so a requirement declared with only such values does not conform. `ratings`, when present, MUST be a map keyed by rating level IDs from the model's rating scale. Each value MUST be a non-empty scalar criterion. The meaning of a criterion override is defined in [Rating scale semantics](#rating-scale-semantics). ## Model semantics The rules in this section define what a conforming model means. They bind every interpretation of the document, by humans or tools, but they do not prescribe any evaluation method, aggregation approach, or report format. ### Source semantics An area that declares `source` selects its own evaluated material. An area that omits `source` inherits the source of the nearest ancestor area that declares one; if no ancestor declares one, it inherits the document's default source defined in [Document structure](#document-structure). An effective source selector identifies the subject or starting boundary an evaluation judges. It does not define an exhaustive prompt payload, evidence bundle, or permission boundary for supporting context. An evaluation method MAY inspect other authorized context when a requirement needs interpretation, comparison, or verification, but it MUST distinguish that supporting context from evidence about the selected subject and MUST NOT silently widen the area or requirement being judged. Tools MAY detect a selector as a path, glob, or prose description for validation, display, and workspace-safety policy. Paths and globs resolve relative to the containing QUALITY.md file. Prose selectors are interpreted by the evaluation method. An absolute filesystem selector or one that escapes the authorized workspace MUST fail containment rather than becoming prose. A tool that cannot inspect the selected subject or required supporting context MUST report that limit instead of evaluating against invented or silently substituted material. ### Requirement scope A requirement is scoped to the area on which it is declared: it states an expectation about that area's source, and only that source. Child areas do not inherit ancestor requirements. Declaring a child area narrows attention to that area's source; it does not re-apply, copy, or extend any ancestor expectation to the child. An ancestor area's source can overlap a descendant area's source. Overlap does not reassign, merge, or deduplicate anything: the ancestor's requirements still read against the ancestor's source, and the descendant's requirements read against the descendant's source. The same material can therefore be subject to requirements declared on more than one area, each read at its own declaring area's scope. ### Factor connection Factor connection relates requirements to the factors that describe them, on one area. A requirement declared under a factor or sub-factor is connected to that factor by placement; the containing factor is its primary factor. Entries in a requirement's `factors` list connect it to further factors declared on the same area as same-area secondary references. A factor's meaning spans, together: * requirements declared under that factor or its sub-factors, and * requirements declared on the same area that reference it explicitly. Factor connection is a many-to-many relation over a single set of requirements. A requirement connected to several factors remains one requirement with one assessment; connection describes it from several quality perspectives, it does not multiply it. ### Rating scale semantics The rating scale's sequence order is part of its meaning: levels are declared from best to worst. A rating level's `description` and `criterion` have distinct semantics. The `description` defines what the level means across the model. The `criterion` defines the default rule for deciding whether a requirement sits at that level. A requirement's `ratings` entry overrides only that level's criterion for that requirement. An override MUST NOT be read as altering a level's `description`, `title`, `level`, or position in the scale order. Rating levels have no meaning beyond what the model declares. Level names such as `target`, `pass`, or `fail` carry no built-in semantics; an interpretation MUST NOT assume fixed rating level meanings the declared scale does not state. ## Body semantics The Markdown body documents context for building, interpreting, using, and evaluating the model. The format does not require any body section names, ordering, or content. The body can document the root area, scope, stakeholder needs, risks, unknowns, open questions, evidence context, or other important context for anyone interpreting or applying the model. ## Extensions A document MAY include frontmatter properties beyond those defined in [Frontmatter schema](#frontmatter-schema). Extension properties MUST NOT change the meaning of properties defined in this specification and should use names unlikely to conflict with future versions of this specification. Note: This note is non-normative. Tools that rewrite QUALITY.md documents are expected to preserve body content and extension properties they do not interpret; dropping or reinterpreting them changes a document the tool did not author. Tools can freely build additional capability on top of the format — filters, output formats, aggregation methods, authoring aids — as long as they read the document according to [Model semantics](#model-semantics). ## Appendix A: Suggested rating scale This appendix is non-normative. When a graded scale fits and an author has no stronger domain-specific scale, a tool can seed the following four-level scale: ```yaml theme={null} ratingScale: - level: outstanding title: Outstanding description: "The stretch band: reached only with significant extra effort." criterion: "Exceeds the requirement; satisfies it with margin to spare." - level: target title: Target description: "The level to aim for: achievable at reasonable cost and effort." criterion: "Satisfies the requirement." - level: minimum title: Minimum description: "The acceptable floor: less than the target, but good enough to rely on." criterion: "Falls short of the target but remains acceptable." - level: unacceptable title: Unacceptable description: "Below the floor: not good enough to rely on." criterion: "Does not meet the requirement to an acceptable degree." ``` ## Appendix B: Minimal example This appendix is non-normative. This is an illustrative software product example, not a default domain or factor set for QUALITY.md. The same model shape applies across domains; only the domain-carried factors, requirements, and assessments change. For a worked non-software example, see [Modeling quality across domains](https://github.com/qualitymd/quality.md/blob/main/docs/guides/model-quality-across-domains.md#worked-example-a-documentation-set). ```markdown theme={null} --- title: Acme Checkout API description: Public API for accepting and settling customer payments. ratingScale: - level: target title: Target description: "Meets the agreed quality bar." criterion: "Satisfies the requirement." - level: unacceptable title: Unacceptable description: "Does not meet the agreed quality bar." criterion: "Does not satisfy the requirement." factors: reliability: title: Reliability description: The API continues to accept and durably record orders. requirements: checkout-requests-durable: title: Checkout requests are durably recorded assessment: Review production write-path telemetry and recovery tests. --- # Acme Checkout API This model covers the checkout API and the payment write path it owns. ``` ## Appendix C: Invalid counter-examples This appendix is non-normative. The following snippets illustrate invalid shapes. They are intentionally incomplete and are not standalone QUALITY.md files. ### Missing rating-level title ```yaml theme={null} ratingScale: - level: target criterion: "Satisfies the requirement." - level: unacceptable title: Unacceptable criterion: "Does not satisfy the requirement." ``` Invalid because every rating level MUST declare `level`, `title`, and `criterion`. ### Direct area requirement without factors ```yaml theme={null} requirements: checkout-requests-durable: title: Checkout requests are durably recorded assessment: Review production write-path telemetry and recovery tests. ``` Invalid because a requirement declared directly under an area MUST declare `factors` with at least one non-empty scalar entry. ### List-valued assessment ```yaml theme={null} factors: reliability: title: Reliability requirements: checkout-requests-durable: title: Checkout requests are durably recorded assessment: - Review telemetry. - Review recovery tests. ``` Invalid because `assessment` MUST be a single non-empty scalar. ## Appendix D: An illustrative evaluation method This appendix is non-normative. QUALITY.md does not prescribe an evaluation method. Any process that reads a model according to [Model semantics](#model-semantics) can evaluate it: a team working through requirements in a review meeting, a scheduled audit, a CI check, or an AI agent. This appendix sketches one concrete method — the one used by the `/quality` agent skill — to illustrate how the declared semantics play out in practice. Nothing in this outline adds conformance requirements. The skill evaluates a model in phases: 1. **Scope.** The evaluation covers the whole model by default, or is narrowed by area or factor. A narrowed evaluation is recorded and reported as scoped, never presented as a whole-model verdict. Each in-scope area's source is interpreted per [Source semantics](#source-semantics). 2. **Assess.** Each in-scope requirement is assessed once, against the source of its declaring area, by applying its `assessment`. This produces evidence-backed findings — gaps, risks, strengths, and notes — along with any unknowns and limits the evaluator hit. 3. **Rate.** Each requirement's findings are judged against each rating level's criterion, with the requirement's `ratings` overrides applied. The outcome is the one level whose criterion the findings satisfy, or `not assessed` when the findings cannot responsibly distinguish the criteria — never a guessed level. 4. **Analyze.** Ratings roll up bottom-up: each factor is characterized from its connected requirements and sub-factors, and each area from its own requirements and its child areas. Every roll-up carries a rationale naming what drove it, and a roll-up over too little assessed material stays `not assessed`. 5. **Advise.** Findings and ratings are distilled into recommendations for improvement or for review of the quality bar itself. Advice never changes a rating. 6. **Report.** The report presents the scope, each requirement's findings and rating with rationale, the factor and area roll-ups, and the advice, with `not assessed` outcomes shown distinctly from rating levels throughout. `not assessed` is this method's own outcome convention, not part of the document format: the skill reserves it so that unjudged material stays visibly distinct from rated material instead of being guessed into a rating level. Other methods are equally legitimate. A method might rate only requirements and skip roll-ups, apply numeric scoring, or re-evaluate a single area on a schedule. What makes any of them an evaluation of the same model is that it reads each requirement against its declaring area's source, honors the declared criteria and scale order, and asserts no meanings the model does not declare.