Autonomous Agents Skill Pack Files

schema_version: 1
id: agent_qa_pass
title: "QA Pass With Handoff"
description: "Multi-session QA pass over a quality epic's task — checkpointed handoff, findings posted on the epic, closing verdict"
overview: |
  The quality lane's multi-session pass. A QA task on a quality epic rarely fits one turn: an area has more
  scenarios than a single session can walk, and a task that seeds fixtures can only be judged once the runs it
  started have finished. This job therefore treats the ticket's own comments as its checkpoint — a handoff block
  from an earlier session names what is already covered and what remains, so the next agent resumes the walk
  instead of repeating it, and a session that finds a verdict already posted re-checks the acceptance criteria
  once and closes the task.

  Defects leave as findings, not as tracker issues (pack 1.32.0). Every defect, observation, stale pack or
  missing fixture profile is a numbered finding (KEY-Fn) posted in one block on the ticket's parent epic —
  before the report, so the gate never sees it — and the epic's own close-out step, run once every QA task
  is done, builds the ledger, merges what shares a root cause, and files the Bugs with the whole picture in
  view. The report on the ticket opens with five fixed header lines a reader and the consolidation step read
  the same way, keeps its result cells neutral (ok, defect (Fn), n/a) and carries the verdict token exactly
  once, on the last line, because the playbook gate scans the whole comment in routing order and a stray
  verdict word inside a table outranks the real one at the end.

category:
  level: pipeline
  domain: engineering

metadata:
  agent: true
  role: qa
  # Gate-verdict vocabulary for portal playbook steps (PLAYBOOKS.md §5) — matched from the
  # step's ticket comment; declaring it here is what makes this workflow content-routable.
  # PASS = the pass is complete (recorded findings never lower it); FAIL = the pass is incomplete or
  # untrustworthy and pauses for a person; BLOCKED pauses. HANDOFF is the session-continuation
  # token: matched as a bare line only and routed through the step's retry set (the portal's
  # CONTINUATION_WORKFLOWS map names this job with it), so the QA Pass seed re-dispatches this
  # same pass to the next session. The prompt below allows it ONLY as the final line of a handoff.
  verdicts: [PASS, FAIL, HANDOFF, BLOCKED]

params:
  agentic_app:
    type: string
    default: "cc"
    description: "Agent-app slug selecting the per-run env file ~/.agent-env.d/<slug> (AAP-C); 'cc'/subscription clears provider vars"
  ticket_url:
    type: string
    description: "Jira ticket URL — the QA task this pass covers"
  depth:
    type: string
    default: "L3"
    description: "Max test depth: L0 (smoke), L1 (structure), L2 (interaction), L3 (data), L4 (edge)"
  hint:
    type: string
    default: ""
    description: "Optional extra instructions for the agent; also carries the CONTINUATION note on a re-dispatch"
  entry_path:
    type: string
    default: "~/workspace"
    description: "Working directory for the agent"

steps:
  - type: terminal.open
    title: "Agent: QA Pass"
    cwd: "{{entry_path}}"
  - type: terminal.run
    cmd: |
      source ~/.agent-env
      # AAP-C (SCRUM-1506) + AAP-17 (SCRUM-1653): clear EVERY provider var an agent-app can deliver so
      # none hijacks/poisons a subscription run. A stray global ANTHROPIC_MODEL / ANTHROPIC_SMALL_FAST_MODEL
      # needs no CLAUDE_CODE_USE_* flag, so the old ${!CLAUDE_CODE_USE_@} glob never caught it — it survived
      # into the run and 404-ed aux/small-fast calls against api.anthropic.com. This explicit list mirrors
      # AGENT_APP_ENV_KEYS 1:1 (the-assistant website/src/lib/cloud/agent-app-env.ts — the single source of
      # truth; a parity test in each repo guards the two from drifting). Explicit over a glob: the union has
      # non-ANTHROPIC_ members (AWS_REGION, AWS_PROFILE, CLOUD_ML_REGION, CLAUDE_CODE_MAX_OUTPUT_TOKENS) and
      # a ${!AWS_@} glob would over-clear unrelated creds. Then source the per-run app env by slug when it
      # exists; no file => subscription/default. base/19-secrets stages ~/.agent-env.d/<slug>.
      unset \
        ANTHROPIC_API_KEY ANTHROPIC_BASE_URL ANTHROPIC_AUTH_TOKEN CCR_CONFIG_B64 \
        CLAUDE_CODE_USE_BEDROCK AWS_REGION AWS_PROFILE ANTHROPIC_MODEL \
        ANTHROPIC_SMALL_FAST_MODEL_AWS_REGION ANTHROPIC_SMALL_FAST_MODEL CLAUDE_CODE_MAX_OUTPUT_TOKENS \
        CLAUDE_CODE_USE_VERTEX CLOUD_ML_REGION ANTHROPIC_VERTEX_PROJECT_ID ANTHROPIC_VERTEX_BASE_URL \
        CLAUDE_CODE_USE_FOUNDRY ANTHROPIC_FOUNDRY_RESOURCE ANTHROPIC_FOUNDRY_BASE_URL \
        ANTHROPIC_DEFAULT_OPUS_MODEL ANTHROPIC_DEFAULT_SONNET_MODEL ANTHROPIC_DEFAULT_HAIKU_MODEL
      if [ -f "$HOME/.agent-env.d/{{agentic_app}}" ]; then
        source "$HOME/.agent-env.d/{{agentic_app}}"
      fi
      claude --dangerously-skip-permissions "$(cat <<'SB_PROMPT'
      read ticket with attachments and all comments - {{ticket_url}}. if it fails, stop and report the error. note the ticket's parent epic: this session's findings are posted there.
      then load the knowledge packs TOP-DOWN before you test anything: the domain root _skill.md, then that domain's qa role file, then the module _skill.md for every module the ticket names AND for every further module you discover is in scope while walking it, then each of those modules' qa.md. walk the module qa.md phases in their given order, up to depth {{depth}} — or deeper when the ticket's description names a deeper level (L0 smoke, L1 structure, L2 interaction, L3 data, L4 edge). a module with no qa.md is tested from its _skill.md Common Tasks instead. a pack that is stale or missing for a module in scope is a finding of kind pack: record it, and test that module against the ticket directly.
      the ticket's scope and acceptance criteria define this pass. they are the baseline you judge against, not your own idea of what the area ought to do: every criterion gets a result, and anything you notice outside them is a finding of kind observation, never a result.
      EXECUTION RULES (read first — they override any habit):
      - Do ALL of the work in THIS session, in the FOREGROUND, and finish before your turn ends. You MAY spawn subagents (the Agent / Task tool) or run parallel searches to investigate faster — but you MUST wait for every subagent and any background task to return and fold their results in YOURSELF before ending the turn. Never move the work (or the comment) into the background — no run_in_background, no background workflows, no trailing & — and then end the turn expecting to be resumed. Ending your turn is FINAL here: nothing wakes you to "resume", so a turn that ends while a spawned subagent or any background work is still pending is reported as a FAILED step, not a pause.
      - Post the ticket comment YOURSELF, in the foreground, as the LAST thing you do — after every subagent and background task has returned. Never delegate that comment to a subagent, and never end the turn "holding", "pausing", or planning to "resume/continue later". If the pass genuinely does not fit this session, that is the handoff shape below: post it and stop. It is never a reason to leave work running past your turn.
      CONTINUATION: if a CONTINUATION note appears in the hint below, or the ticket carries a handoff comment from a previous session, this task is already mid-pass — read that checkpoint FIRST and continue from it. Do NOT restart the pass and do NOT re-run a scenario it already recorded a result for; walk what its checklist lists as remaining, plus anything its evidence shows it could not reach. If the checkpoint is unusable, say why in your comment before you start over.
      CLOSING SESSION: if the ticket already carries a comment from an earlier session of this same task whose last line is a verdict line (Verdict: PASS or Verdict: FAIL), this is the closing session — after fixes landed, or after a person resumed a pass that was left incomplete. Do not re-walk the area: the recorded defects are this task's findings on the epic and the Bugs the epic's consolidation linked to this ticket; re-check those ONCE against what landed since, complete whatever the earlier session reported as incomplete, then post the final verdict for the task. if the ticket carries BOTH a handoff comment and a verdict line, the LATER of the two decides which rule you are under: a verdict posted after the last handoff makes this the closing session, and a handoff posted after the last verdict means the closing re-check is itself mid-pass — continue that re-check from its checkpoint.
      WAITING: when the only work left is waiting for runs or jobs you started to reach a state, wait for them IN THIS SESSION — poll every two to three minutes, for up to forty minutes — then judge them. A handoff is never a way to wait: the platform re-dispatches the next session within a minute, so a handoff posted in order to wait burns a session and lands in the same wait again. Hand off only when you must stop for another reason while they are still running, and say in the block exactly which ids the next session must still wait for.
      HARNESS: bring up the lane the domain pack prescribes, on your own scratch data — never production, and never another tenant's workspace, run, job, artifact or ticket: someone else's data is a leak, not evidence. the fixture is a recipe, not shared data: run the fixture recipe the epic or the pack names on your scratch lane, with the profile your area needs when the recipe offers one; extend it locally for what the recipe lacks and record the extension in your results artifact so the next session can rebuild the same state. a recipe or profile the area needed and did not have is a finding of kind harness. when the ticket's own environment line cannot be honoured, say so in one sentence on your report's Harness line and go on.
      FINDINGS: this job files NO tracker issues. every defect is a numbered finding KEY-Fn — KEY is this ticket's key, n continues across this task's sessions (read the epic's earlier blocks for this task to find the last n) — carrying: severity P0–P3, kind (defect / observation / pack / harness), the module or area, a one-line symptom described rather than labelled, numbered repro steps, expected vs actual, the exact URL at the time of failure, the published evidence link, and the pointer when the ticket or its design doc predicted it. an observation carries what you saw and where; no repro is owed. read the titles of the epic's earlier blocks once before posting and add a "same as" hint where another task already recorded the symptom — a hint, not a decision: consolidation merges, you never do.
      EPIC: post ALL of this session's findings, in full, as ONE comment on the ticket's parent epic — headed "QA findings · KEY · session n", sections Defects / Observations / Pack and harness / Coverage, where Coverage is one line (modules · depth reached · scenarios · ok · defect · n/a) plus the link to this session's results artifact — and post it even when the session found nothing. post it BEFORE your ticket comment; over 30,000 characters split it into numbered parts; if the ticket has no parent, post the same block on the ticket itself, before the report. a P0 finding — data loss, script execution, an auth or tenant boundary crossed — is posted on the epic the moment it is confirmed, as its own block headed "QA findings · KEY · session n · early", under the id the final block repeats. a session that hands off, or is blocked after walking anything, posts its block first too. the verdict words, HANDOFF and BLOCKED appear nowhere in an epic block.
      REPORTING DISCIPLINE — the playbook gate string-matches your WHOLE ticket comment, not just its last line, and it tests the verdicts in routing order (retry → loopback → pass → stop → pause), so one stray token anywhere in the body outranks the genuine one at the end. Report every scenario result NEUTRALLY, as one of three cell values: ok (behaved as the acceptance criteria say), defect (cite the finding id: defect (F3)) or n/a (not applicable, or not reachable — say which). Never the bare words pass/fail/passed/failed, in any case, in a cell, a heading or a sentence, and never the word blocked. Describe a finding, do not label it: "the priority select keeps its old value after save", not "priority: failed". Keep the verdict token out of the body entirely, and the word verdict itself too — both appear exactly once, on the last line.
      BUDGET: check the clock. a session past about two hours with scope still remaining hands off rather than grinds; the next session starts from your checkpoint at no loss.
      {{hint}}
      FINISH — your LAST action is exactly ONE ticket comment, in exactly ONE of these three shapes:
      1. REPORT — this session completed the pass, or closed it out. its first five lines are fixed, one per line: "QA · KEY · session n" · "Harness: <repository @ commit · lane · fixture recipe and extension · account / workspace>" · "Scope: <modules · depth reached · scenarios walked · ok · defect · n/a>" · "Findings: <d defects · o observations · p pack · h harness → epic KEY>" · "Results: <link>". then the coverage matrix (module or area × depth level, cells ok / defect (Fn) / n/a), the findings table (id · severity · kind · symptom — no tracker keys, none exist yet), the evidence links, and as the LAST line either "Verdict: PASS" — the pass is COMPLETE: every scenario in scope walked to depth {{depth}}, every result recorded, every finding posted on the epic; findings never lower it, the epic's consolidation follows each one up — or "Verdict: FAIL" — the pass is INCOMPLETE or its results cannot be trusted: coverage you could not reach and cannot hand off, a harness you could not make valid, acceptance criteria that cannot be verified at all. FAIL pauses the task for a person, so name exactly what a person must supply or decide. the per-scenario table lives in the results artifact and is repeated here only when it has thirty rows or fewer. The tokens HANDOFF and BLOCKED appear nowhere in this shape.
      2. HANDOFF — the pass does not fit this session (for runs still in flight see WAITING): FIRST publish the evidence and the results artifact for everything you did cover and post the epic block for it, THEN post a block carrying what is done (the scenarios walked, with their results), what remains (a concrete checklist of the scenarios still to walk), the single next step the next session starts from, the evidence links, and the run and job ids anything in flight is waiting on. End the comment with the token HANDOFF alone on its own last line. The platform dispatches another session to continue from that block. Write no verdict line in this shape.
      3. BLOCKED — a missing decision, credential, access or environment stops the pass entirely: state what blocks it and what would unblock it. End the comment with the token BLOCKED alone on its own last line. Write no verdict line in this shape.
      Never mix the shapes: HANDOFF and BLOCKED are routing tokens — each may appear ONLY as the final line of its own shape, and a report contains neither.
      EVIDENCE: publish as you go, not at the end. Run publish_artifact per test step for the screenshot or extract that step produced — a file that evidences a finding carries the finding id first in its name (KEY-F3-…) — and once per session for the results artifact KEY-qa-results-n.md (n is this session's number on the task), which carries the harness, the full per-scenario table, the fixture extension and every finding in full; cite the returned download links in the comments above. If publish_artifact is unavailable, save every file under ~/workspace/artifacts/ for post-run collection, as in the other QA jobs. Keep evidence filenames free of verdict words.
      UNATTENDED RUN — this is a dispatched job: nobody is watching this terminal and nobody will answer, so never call AskUserQuestion and never end your turn by putting a question to the operator. Apply the policy stated above and carry it out as written, including where carrying it out is destructive — the call to run this job was made when it was dispatched, and it is not yours to re-take. Where that policy leaves a genuine choice, make it yourself and keep going; only where it is genuinely SILENT, prefer the option that does not destroy work nobody can recreate. Say what you decided and what you skipped and why — but INSIDE the report this prompt already specifies: wherever a report shape, a token vocabulary or a last-line verdict is stated above, that format still binds exactly as written, and this clause adds no section, no token and nothing after the last line. Questions the deliverable is MEANT to contain are content, not an unanswered prompt. Asking does not pause the job either: an unanswered prompt is auto-answered from the option ordering once the unattended wait budget lapses (KAN-204), so asking only burns ~15 minutes of the run and hands a real decision to a default.
      SB_PROMPT
      )"