Add agent-cost-evidence-audit skill

[linqiang-max]

Summary

Adds an offline skill for reviewing whether local LLM and agent run data supports cost, cost-per-success, retry, fallback, cache, provider-attribution, and billing claims.

The skill includes:

• a focused evidence-review workflow;
• a standard-library-only Python validator;
• metadata-only synthetic regression cases covering provider attribution, cache overlap, aggregate-to-call fabrication, task acceptance, retry/fallback evidence, unpriced usage, and estimated-versus-billed cost.

User value

Existing instrumentation and cost guidance can help collect traces or calculate prices. This skill addresses a different failure mode: a numerically plausible result can still be unsupported because the source data lacks a hosting provider, call boundary, acceptance result, or billing evidence. The validator makes those semantic checks deterministic and suitable for local or CI use without an API key or paid service.

Privacy and dependencies

• No network calls, credentials, prompts, completions, or personal data.
• Synthetic fixture only.
• Python standard library only.
• No pricing catalog or vendor-specific rate table.

Validation

• npm run skill:validate
• npx --yes @microsoft/vally-cli lint skills/agent-cost-evidence-audit --verbose
• python skills/agent-cost-evidence-audit/scripts/audit_agent_cost_evidence.py --self-test

Scope

This skill audits evidence sufficiency. It does not calculate provider prices, reconstruct invoices, infer missing providers, or claim compatibility with every trace format.

License

The human author approved MIT contribution of exactly these three original files: SKILL.md, fixtures/synthetic-audit-cases.json, and scripts/audit_agent_cost_evidence.py. The generated docs/README.skills.md row repeats the approved SKILL description and is produced by the upstream build. No Website, Dataset, private pricing engine, or research-package file is included.

[github-actions]

🔒 PR Risk Scan Results

Scanned 3 changed file(s).

Severity	Count
🔴 High	0
🟠 Medium	0
ℹ️ Info	1

Severity	Rule	File	Line	Match
ℹ️	skill-script-touched	skills/agent-cost-evidence-audit/scripts/audit_agent_cost_evidence.py	1	skills/agent-cost-evidence-audit/scripts/audit_agent_cost_evidence.py

This is an automated soft-gate report. Findings indicate review targets and do not block merge by themselves.

[github-actions]

🚦 Submission status: 👀 Ready for review

Risk tier: merge-risk:high — Privileged execution, automation, or review-policy change
Required to merge: passing submission-gate checks plus 1 approval from reviewers with write access, including a maintainer with admin or maintain permission.

Why this tier

• skills/agent-cost-evidence-audit/scripts/audit_agent_cost_evidence.py is a high-risk path (automation, scripts, MCP config, hooks, or review policy)

Automated checks

Check	Status	Details
Line endings	✅ Passed	Passed · logs
Spelling	✅ Passed	Passed · logs
Generated README consistency	✅ Passed	Passed · logs
Skill validation	✅ Passed	Passed · logs
Skill lint (vally)	✅ Passed	Passed · logs
Risk scan	✅ Passed	Passed · logs
Contributor reputation	✅ Passed	Passed · logs
Duplicate resource scan	✅ Passed	Passed · logs
PR quality signal	⏭️ Skipped	Skipped by its workflow · logs

Review

• Approvals: 0/1
• Assigned reviewer: not assigned yet — comment /request-review to ask for one
• Review target date: not set
• Still needed: 1 more approval(s); an approval from a maintainer with admin or maintain permission
• The core-maintainers pool is not staffed yet; an approver with admin or maintain permission is required instead.

Commands

Command	Who	What it does
/rerun-checks	PR author, maintainers	Re-runs failed or incomplete checks and re-evaluates this gate
/request-review	PR author, maintainers	Asks the review rotation to assign a reviewer (adds needs-reviewer)

_{Updated for 7bf2445 · gate run · This comment is maintained automatically — see submission gate docs.}

[github-actions]

🔍 Vally Lint Results

✅ All checks passed

Scope	Checked
Skills	1
Agents	0
Total	1

Severity	Count
❌ Errors	0
⚠️ Warnings	0
ℹ️ Advisories	0

Summary

Level	Finding
ℹ️	✅ agent-cost-evidence-audit (2/2 checks passed)
ℹ️	✓ [spec-compliance] All 1 skill(s) are spec-compliant.
ℹ️	✓ spec-compliance: All spec checks passed.
ℹ️	✓ [valid-refs] All file references across 1 skill(s) are valid.
ℹ️	✓ valid-refs: All file references resolve to existing files within the skill directory.
ℹ️	1 skill(s) linted, 1 passed

Full linter output

### Linting skills/agent-cost-evidence-audit
✅ agent-cost-evidence-audit (2/2 checks passed)
    ✓ [spec-compliance] All 1 skill(s) are spec-compliant.
        ✓ spec-compliance: All spec checks passed.
    ✓ [valid-refs] All file references across 1 skill(s) are valid.
        ✓ valid-refs: All file references resolve to existing files within the skill directory.

1 skill(s) linted, 1 passed

[linqiang-max]

/request-review

[github-actions]

🙋 Added needs-reviewer. Reviewer routing is assigning a reviewer now.