Discover Skills from Files — Autonomous Agents Agentic Workflow

schema_version: 1
id: agent_sd_files
title: "Discover Skills from Files"
description: "Turn a set of documents into knowledge-pack modules — the sd role's file harness, run over a file set instead of a repo"
overview: |
  The Skill Discovery agent runs this job over a **file set** rather than a set of repositories: a handbook,
  a price list, a policy folder, an export, a synced Drive folder. It reads each document through the File
  Discovery Harness in the sd role playbook — the extracted text is the snapshot, a heading path plus page
  anchor is the selector — groups the set into modules, and fills the same Fill Targets ladder every other
  discovery run fills, so the result is ordinary pack modules that SE, QA and chat agents read like any other.

  It is a separate workflow from `agent_sd_coverage` rather than a fourth mode on it because every mode there
  begins at README-first target resolution — read the repo's README, take the URL it documents or follow its
  setup to bring the product up — and a file set has no README, no deployed URL and nothing to run. The files
  are the target, so this prompt resolves nothing and starts at the set it was given.

  This is an **enrichment, not an ingestion path.** A portal that accepts a file already extracts, chunks,
  summarises and indexes it in process and answers from it on the next question, with no job dispatched and no
  VM involved. This run exists for the other case — messy material worth turning into reusable modules — and
  what it emits carries `confidence` frontmatter like every other module, so it lands in the existing skill
  recompute and renders on the existing Skills pages with no new display code. Material the harness cannot
  fully read is capped and marked rather than rounded up, the same honesty rule a mobile app without an
  emulator gets.

category:
  level: pipeline
  domain: engineering

metadata:
  agent: true
  role: sd
  # No `verdicts`: like `agent_sd_coverage` this is a free-form discovery job whose result is modules in a
  # pack, not a routable outcome. A caller that wants to gate on it reads the pack, not a comment token.

params:
  agentic_app:
    type: string
    default: "cc"
    description: "Agent-app slug selecting the per-run env file ~/.agent-env.d/<slug> (AAP-C); 'cc'/subscription clears provider vars"
  files:
    type: string
    default: ""
    description: "The file set to discover: a folder, a glob, or a comma-separated list of paths on this VM (empty = every document under <entry_path>/files)"
  domain:
    type: string
    default: ""
    description: "Pack domain the modules are emitted under — the directory in the account registry (empty = derive one from the set and say which you chose)"
  hint:
    type: string
    default: ""
    description: "Optional extra instructions — which documents matter, what the material is for, a size or count budget"
  entry_path:
    type: string
    default: "~/workspace"
    description: "Working directory for the agent"

steps:
  - type: terminal.open
    title: "Agent: SD File Discovery"
    cwd: "{{entry_path}}"
  - type: terminal.run
    cmd: |
      source ~/.agent-env
      # AAP-C (SCRUM-1506) + AAP-17 (SCRUM-1653): clear EVERY provider var an agent-app can deliver so
      # none hijacks/poisons a subscription run. A stray global ANTHROPIC_MODEL / ANTHROPIC_SMALL_FAST_MODEL
      # needs no CLAUDE_CODE_USE_* flag, so the old ${!CLAUDE_CODE_USE_@} glob never caught it — it survived
      # into the run and 404-ed aux/small-fast calls against api.anthropic.com. This explicit list mirrors
      # AGENT_APP_ENV_KEYS 1:1 (the-assistant website/src/lib/cloud/agent-app-env.ts — the single source of
      # truth; a parity test in each repo guards the two from drifting). Explicit over a glob: the union has
      # non-ANTHROPIC_ members (AWS_REGION, AWS_PROFILE, CLOUD_ML_REGION, CLAUDE_CODE_MAX_OUTPUT_TOKENS) and
      # a ${!AWS_@} glob would over-clear unrelated creds. Then source the per-run app env by slug when it
      # exists; no file => subscription/default. base/19-secrets stages ~/.agent-env.d/<slug>.
      unset \
        ANTHROPIC_API_KEY ANTHROPIC_BASE_URL ANTHROPIC_AUTH_TOKEN CCR_CONFIG_B64 \
        CLAUDE_CODE_USE_BEDROCK AWS_REGION AWS_PROFILE ANTHROPIC_MODEL \
        ANTHROPIC_SMALL_FAST_MODEL_AWS_REGION ANTHROPIC_SMALL_FAST_MODEL CLAUDE_CODE_MAX_OUTPUT_TOKENS \
        CLAUDE_CODE_USE_VERTEX CLOUD_ML_REGION ANTHROPIC_VERTEX_PROJECT_ID ANTHROPIC_VERTEX_BASE_URL \
        CLAUDE_CODE_USE_FOUNDRY ANTHROPIC_FOUNDRY_RESOURCE ANTHROPIC_FOUNDRY_BASE_URL \
        ANTHROPIC_DEFAULT_OPUS_MODEL ANTHROPIC_DEFAULT_SONNET_MODEL ANTHROPIC_DEFAULT_HAIKU_MODEL
      if [ -f "$HOME/.agent-env.d/{{agentic_app}}" ]; then
        source "$HOME/.agent-env.d/{{agentic_app}}"
      fi
      claude --dangerously-skip-permissions "$(cat <<'SB_PROMPT'
      Learn the Skill Discovery methodology before you open a file: load the sd role playbook — its File Discovery Harness, its Fill Targets ladder and the SD Frontmatter Contract.

      This run is scoped to a FILE SET, not to repositories: {{files}}. If that is empty, take every document under {{entry_path}}/files.
      TARGET RESOLUTION DOES NOT APPLY. Do not look for a README, a deployed URL, a docker compose file or a dev server, and do not try to bring any product up — the files ARE the target and the File Discovery Harness is the harness. Resolve nothing; inventory the set instead.

      INVENTORY: list the set — path, format, size, revision or issue date, language — and group it into modules. A module is one document, or one coherent group of documents answering the same kind of question; a document whose top-level chapters serve different audiences splits into one module per chapter. Name modules from the file name or the document's own title. Read each document IN FULL through its extractor before you write anything down (markdown/text/CSV/HTML as-is, office formats through an office extractor, PDFs through pdftotext -layout), and record the extractor and its flags in the module — a different extractor yields different anchors.

      FILL: the pack domain for this run is '{{domain}}'. If that is empty, resolve the domain FIRST (see ROOT below) and use the name you chose everywhere below — never a bare leading '/', which is the filesystem root. For each module write <domain>/<module>/_skill.md against the ladder, using the file analogs the harness defines — document identity for URL Patterns, the heading tree for Page Structure, an anchors table (passage | heading path + page | notes) for Key Elements, the entities the document defines for Data Model, its editions and validity windows for States, the questions it answers with the heading path to each answer for Common Tasks, and extraction pitfalls for Tips and Gotchas. Every anchor is re-derived against a fresh extraction before you record it; prefer the heading path over the page number, which moves when a document is re-laid out.

      CONFIDENCE — nothing rounds up. A document you could not fully read caps its module at 25% with the affected sections marked harness-blocked and the reason named (no_text_layer, encrypted, tables_flattened, unsupported_format, truncated). A partially read document reads as partially read: the number's only job is to tell an operator which material their agents cannot actually answer from. Every _skill.md you write or touch carries a numeric `confidence` in its frontmatter on the 0.0-1.0 scale — 25% is `0.25`, 1% is `0.01`, never 25 or 1. That is what the portal's skill recompute reads; it parses the number as written and clamps nothing, so a percentage written whole renders as 2500% and promotes the whole domain to the top readiness band. A module without the field is counted nowhere.

      VISIBILITY: material an admin uploaded is internal — the model may read it, an end user may not be handed it. A module derived from it inherits that: publish to the ACCOUNT registry only, never the public catalog, and quote only what a reader of the module needs instead of reproducing the document. Say under What This Is whether a module came from public or from internal material.

      ROOT: update the domain root <domain>/_skill.md — a Target Resolution block naming the file set, the extractors used and the date, a Module Inventory row per module with its coverage, and the portal-linking frontmatter (workspaces:/repos:) per the SD Frontmatter Contract. If the run was given no domain, choose one from the set, create it, and say in your report which one you chose and why. A domain you create is not installable until it also carries its own skill-pack.json AND a matching row in the registry's index.json — without both, the portal counts the modules but `sidebutton registry update` skips them and no skill:// URI ever resolves, so write both and run the registry's own scripts/check.mjs before you push.

      BUDGET: work document by document, deepest material first. If the set is larger than the session, finish the modules you started, leave the rest listed in the Module Inventory at 1%, and name in your report exactly what was not reached — never a half-written module and never an inflated confidence.

      ENRICHMENT, NOT INGESTION: nothing downstream waits on this run. A portal that accepts these files already extracts, chunks, summarises and indexes them in process and answers from them without any job — this run turns messy material into reusable pack modules beside that pipeline, so never modify, move, rename or delete a source document, and never treat this job as a step some upload depends on.

      Install changes with sidebutton registry update, then publish ONLY to the registry SideButton configured from the SIDEBUTTON_DEFAULT_REGISTRY env var (find it with sidebutton registry list): edit and commit on a branch in the registry clone at ~/.sidebutton/registries/<name> (auth is already on the VM). Then PUBLISH SO YOUR WORK REACHES THE MAIN BRANCH (main might be "master").
      Do NOT commit, push, or PR to any repo other than that one. If sidebutton registry list shows no writable account registry, STOP and report.
      {{hint}}
      Report the modules created or improved, each one's confidence and what capped it, the documents you could not read and why, and anything in the set you did not reach.
      UNATTENDED RUN — this is a dispatched job: nobody is watching this terminal and nobody will answer, so never call AskUserQuestion and never end your turn by putting a question to the operator. Apply the policy stated above and carry it out as written, including where carrying it out is destructive — the call to run this job was made when it was dispatched, and it is not yours to re-take. Where that policy leaves a genuine choice, make it yourself and keep going; only where it is genuinely SILENT, prefer the option that does not destroy work nobody can recreate. Say what you decided and what you skipped and why — but INSIDE the report this prompt already specifies: wherever a report shape, a token vocabulary or a last-line verdict is stated above, that format still binds exactly as written, and this clause adds no section, no token and nothing after the last line. Questions the deliverable is MEANT to contain are content, not an unanswered prompt. Asking does not pause the job either: an unanswered prompt is auto-answered from the option ordering once the unattended wait budget lapses (KAN-204), so asking only burns ~15 minutes of the run and hands a real decision to a default.
      SB_PROMPT
      )"