Validate Task Resolution — Autonomous Agents Agentic Workflow

schema_version: 1
id: agent_ops_validate_resolution
title: "Validate Task Resolution"
description: "Read-only: after a ticket goes Done, judge whether the outcome it was opened to deliver actually landed — verified against the repo, not just the comments — and post one verdict comment"
overview: |
  The post-Done resolution check (Task Validation V2, SCRUM-1634). After a ticket's playbook run has
  finished, this PM workflow re-reads the ticket — its acceptance criteria AND every comment,
  including the prior runs' own gate comments ("PASS", "merged", "QA PASS", …) — and the repository,
  then judges one question: was the OUTCOME this ticket was opened to deliver actually delivered,
  verified against the code and not just the comments? It exists because gates judge step advancement
  and QA workflows test behaviour, but nothing re-reads the ticket after Done against what landed.

  It judges delivery, not tracker hygiene. Unfiled follow-ups, stale description text on a re-scoped
  ticket, operator-accepted deferrals and in-thread findings belonging to other tickets are reported
  under Notes and never move the verdict — the first prompt made them gaps, and the operator dismissed
  55% of the INCOMPLETE verdicts that resulted (116 of 211 in the first three weeks of prod).

  It is strictly recording-only. The agent reads the ticket and the repo, confirms the PRs/commits the
  comments name really exist and really contain the claimed change, and posts ONE verdict comment:
  VALIDATED, INCOMPLETE (with a numbered gap list) or UNVERIFIABLE. It recommends only — it files
  nothing, changes no code, merges nothing, and never transitions the ticket. The engine records the
  verdict; a human acts on it.

  The comment's SHAPE is a contract with the portal, not styling — see ops/_skill.md "Validation
  Verdict Comment Contract". The gap list is machine-sliced into the re-run steering hint
  (`extractValidationGap`, the-assistant website/src/lib/tasks/scheduler.ts), and the verdict's
  timestamp is stamped and displayed portal-side (`evaluated_at_ms`, Tasks band 1), so the agent must
  not date its own comment.

category:
  level: pipeline
  domain: engineering

metadata:
  agent: true
  # SINGLE SOURCE for the validation dispatch role (2026-07-20 ops→pm move): the portal resolves
  # this field through the synced workflow registry at enqueue — nothing portal-side hardcodes it,
  # so changing it here (+ version bump) IS the cutover. Dispatch eligibility does not depend on
  # it: task_validation-tagged items match any agent (TASK-RUN-VALIDATION.md §5.4); the role
  # drives queue/jobs badges and per-role KPI attribution.
  role: pm
  # Gate-verdict vocabulary for portal playbook steps (PLAYBOOKS.md §5) — matched from the
  # step's ticket comment; declaring it here is what makes this workflow content-routable.
  verdicts: [VALIDATED, INCOMPLETE, UNVERIFIABLE]

params:
  agentic_app:
    type: string
    default: "cc"
    description: "Agent-app slug selecting the per-run env file ~/.agent-env.d/<slug> (AAP-C); 'cc'/subscription clears provider vars"
  ticket_url:
    type: string
    description: "Jira ticket URL (the Done ticket whose resolution is being validated)"
  hint:
    type: string
    default: ""
    description: "Optional extra instructions for the agent"
  entry_path:
    type: string
    default: "~/workspace"
    description: "Working directory for the agent"

steps:
  - type: terminal.open
    title: "Agent: Validate Task Resolution"
    cwd: "{{entry_path}}"
  - type: terminal.run
    cmd: |
      source ~/.agent-env
      # AAP-C (SCRUM-1506) + AAP-17 (SCRUM-1653): clear EVERY provider var an agent-app can deliver so
      # none hijacks/poisons a subscription run. A stray global ANTHROPIC_MODEL / ANTHROPIC_SMALL_FAST_MODEL
      # needs no CLAUDE_CODE_USE_* flag, so the old ${!CLAUDE_CODE_USE_@} glob never caught it — it survived
      # into the run and 404-ed aux/small-fast calls against api.anthropic.com. This explicit list mirrors
      # AGENT_APP_ENV_KEYS 1:1 (the-assistant website/src/lib/cloud/agent-app-env.ts — the single source of
      # truth; a parity test in each repo guards the two from drifting). Explicit over a glob: the union has
      # non-ANTHROPIC_ members (AWS_REGION, AWS_PROFILE, CLOUD_ML_REGION, CLAUDE_CODE_MAX_OUTPUT_TOKENS) and
      # a ${!AWS_@} glob would over-clear unrelated creds. Then source the per-run app env by slug when it
      # exists; no file => subscription/default. base/19-secrets stages ~/.agent-env.d/<slug>.
      unset \
        ANTHROPIC_API_KEY ANTHROPIC_BASE_URL ANTHROPIC_AUTH_TOKEN CCR_CONFIG_B64 \
        CLAUDE_CODE_USE_BEDROCK AWS_REGION AWS_PROFILE ANTHROPIC_MODEL \
        ANTHROPIC_SMALL_FAST_MODEL_AWS_REGION ANTHROPIC_SMALL_FAST_MODEL CLAUDE_CODE_MAX_OUTPUT_TOKENS \
        CLAUDE_CODE_USE_VERTEX CLOUD_ML_REGION ANTHROPIC_VERTEX_PROJECT_ID ANTHROPIC_VERTEX_BASE_URL \
        CLAUDE_CODE_USE_FOUNDRY ANTHROPIC_FOUNDRY_RESOURCE ANTHROPIC_FOUNDRY_BASE_URL \
        ANTHROPIC_DEFAULT_OPUS_MODEL ANTHROPIC_DEFAULT_SONNET_MODEL ANTHROPIC_DEFAULT_HAIKU_MODEL
      if [ -f "$HOME/.agent-env.d/{{agentic_app}}" ]; then
        source "$HOME/.agent-env.d/{{agentic_app}}"
      fi
      claude --dangerously-skip-permissions "$(cat <<'SB_PROMPT'
      read the ticket with attachments and ALL comments - {{ticket_url}}. if it fails, stop and report the error.
      READ-ONLY pass. the ticket is already Done. judge ONE question: was the outcome this ticket was opened to deliver actually delivered?
      start from context, not a checklist — what problem was filed, what the thread agreed to deliver (the ACs, plus any re-scope the operator accepted in-thread), what the reporter would check to call it solved. verify that in the repo under {{entry_path}}: each PR/commit named exists and contains the claimed change, and any migration/deploy/config an outcome depends on was applied, not just written. comments — the delivery run's own gate comments included — are claims to verify, not evidence to trust.
      these ARE gaps: SCOPED PASS — an approval silently covering only part ("PASS - scoped to Slice A", "code-level only", "not closeable pending X"); MASKED BLOCK — a green headline over an unresolved blocker in the same thread. the body is the truth, not the headline.
      these are NOT gaps: follow-up tickets nobody filed (unless filing them was itself an AC); stale description text on a re-scoped ticket; work the thread deferred, descoped, or accepted with known gaps; bugs belonging to other tickets; timing or process complaints. they go under Notes and never change the verdict.
      {{hint}}
      decide EXACTLY ONE verdict:
        - VALIDATED: the outcome is delivered and verified in the repo. unfiled follow-ups and paperwork do not block this.
        - INCOMPLETE: something the ticket asked for is missing, broken, or unproven in the repo or the environment — the reporter would still see the original problem.
        - UNVERIFIABLE: you could not check (repo unavailable, no stated intent, evidence inaccessible) — say what blocked you.
      recommend only: change no code, touch no PR, change no ticket status, create no issue, post no second comment.
      write EXACTLY ONE comment, as your FINAL action, under 400 words, in THIS order — the portal parses it:
        1. first line EXACTLY: "Validation verdict: VALIDATED" (or INCOMPLETE / UNVERIFIABLE) — that word appears here and NOWHERE else.
        2. evidence: one line per outcome checked -> the PR/commit/file delivering it. don't restate the thread; don't date the comment — the portal stamps and displays the verdict time.
        3. optional "Notes:" — the not-a-gap items, one line each.
        4. INCOMPLETE only, LAST: a line reading exactly "Gaps:", then a NUMBERED list — what is missing -> where you looked -> what would close it. no other line may begin with "gap", and nothing follows the list: the portal slices heading-to-end into the re-run hint.
      quoting the delivery run's own "PASS"/"FAIL"/"merged" prose as evidence is fine — not this step's vocabulary — but invent no new verdict-looking tokens.
      SB_PROMPT
      )"