Regression Test (Live Site) — Autonomous Agents Agentic Workflow

schema_version: 1
id: agent_qa_regression
title: "Regression Test (Live Site)"
description: "Regression test affected modules on the live/deployed site after merge"
overview: |
  After a merge ships to production, this chain re-runs targeted regression checks on the modules the PR touched, plus any modules declared as adjacent in the knowledge pack. The agent loads the module knowledge packs, walks the known-good user paths, and flags any selector drift or behavioural regression it finds.

  This differs from first-pass QA in that it is not driven by acceptance criteria — the baseline is "everything the module used to do still works." Run it automatically after each merge window, or manually whenever a deploy is suspected of breaking unrelated flows.

category:
  level: pipeline
  domain: engineering

metadata:
  agent: true
  role: qa
  # Gate-verdict vocabulary for portal playbook steps (PLAYBOOKS.md §5) — matched from the
  # step's ticket comment; declaring it here is what makes this workflow content-routable.
  verdicts: [PASS, FAIL]

params:
  agentic_app:
    type: string
    default: "cc"
    description: "Agent-app slug selecting the per-run env file ~/.agent-env.d/<slug> (AAP-C); 'cc'/subscription clears provider vars"
  ticket_url:
    type: string
    description: "Jira ticket URL"
  depth:
    type: string
    default: "L3"
    description: "Max test depth: L0 (smoke), L1 (structure), L2 (interaction), L3 (data)"
  hint:
    type: string
    default: ""
    description: "Optional extra instructions for the agent"
  entry_path:
    type: string
    default: "~/workspace"
    description: "Working directory for the agent"

steps:
  - type: terminal.open
    title: "Agent: QA Regression (Live)"
    cwd: "{{entry_path}}"
  - type: terminal.run
    cmd: |
      source ~/.agent-env
      # AAP-C (SCRUM-1506) + AAP-17 (SCRUM-1653): clear EVERY provider var an agent-app can deliver so
      # none hijacks/poisons a subscription run. A stray global ANTHROPIC_MODEL / ANTHROPIC_SMALL_FAST_MODEL
      # needs no CLAUDE_CODE_USE_* flag, so the old ${!CLAUDE_CODE_USE_@} glob never caught it — it survived
      # into the run and 404-ed aux/small-fast calls against api.anthropic.com. This explicit list mirrors
      # AGENT_APP_ENV_KEYS 1:1 (the-assistant website/src/lib/cloud/agent-app-env.ts — the single source of
      # truth; a parity test in each repo guards the two from drifting). Explicit over a glob: the union has
      # non-ANTHROPIC_ members (AWS_REGION, AWS_PROFILE, CLOUD_ML_REGION, CLAUDE_CODE_MAX_OUTPUT_TOKENS) and
      # a ${!AWS_@} glob would over-clear unrelated creds. Then source the per-run app env by slug when it
      # exists; no file => subscription/default. base/19-secrets stages ~/.agent-env.d/<slug>.
      unset \
        ANTHROPIC_API_KEY ANTHROPIC_BASE_URL ANTHROPIC_AUTH_TOKEN \
        CLAUDE_CODE_USE_BEDROCK AWS_REGION AWS_PROFILE ANTHROPIC_MODEL \
        ANTHROPIC_SMALL_FAST_MODEL_AWS_REGION ANTHROPIC_SMALL_FAST_MODEL CLAUDE_CODE_MAX_OUTPUT_TOKENS \
        CLAUDE_CODE_USE_VERTEX CLOUD_ML_REGION ANTHROPIC_VERTEX_PROJECT_ID ANTHROPIC_VERTEX_BASE_URL \
        CLAUDE_CODE_USE_FOUNDRY ANTHROPIC_FOUNDRY_RESOURCE ANTHROPIC_FOUNDRY_BASE_URL \
        ANTHROPIC_DEFAULT_OPUS_MODEL ANTHROPIC_DEFAULT_SONNET_MODEL ANTHROPIC_DEFAULT_HAIKU_MODEL
      if [ -f "$HOME/.agent-env.d/{{agentic_app}}" ]; then
        source "$HOME/.agent-env.d/{{agentic_app}}"
      fi
      claude --dangerously-skip-permissions "$(cat <<'SB_PROMPT'
      read ticket with attachments and all comments - {{ticket_url}}. if it fails, stop and report the error.
      identify which modules the ticket affects from description, PR links, and code changes.
      learn how to QA relevant modules by loading skill packs.
      run regression tests up to depth {{depth}} following the module qa.md test phases in order.
      collect screenshot evidence at phase boundaries.
      {{hint}}
      save each evidence file, publish it via the publish_artifact tool, and cite the returned download link inline in your single ticket comment; if publish_artifact is unavailable, save under ~/workspace/artifacts/ for post-run collection.
      write one comment to the ticket with: modules tested, phases completed, per-phase results (test#, result, notes), new bugs found (repro steps), and overall verdict: PASS (no regressions) or FAIL.
      SB_PROMPT
      )"