Repository navigation
feat(harness): opt-in risk classifier behind DEEPCODE_RISK_CLASSIFIER - #263
Open
raymondginger2018-sudo wants to merge 4 commits into
Open
raymondginger2018-sudo wants to merge 4 commits into
raymondginger2018-sudo wants to merge 4 commits into
Conversation
Prompt-injection defense: DEEPCODE_RISK_CLASSIFIER-gated classifier that scores incoming user/harness content before it reaches the loop. All new files; existing behavior untouched when the knob is off.
Add AgentRunSpec.risk_classifier seam: in _check_permission an 'ask' with no approval_callback (headless) now consults the optional classifier and auto-allows only a decisive 'low' verdict; medium/high/error/no-verdict keep the existing deny — it can upgrade an ask, never widen a deny or escalate risk. build_risk_classifier(provider) returns the env-gated (DEEPCODE_RISK_CLASSIFIER=1) async resolver; off by default (P0-1 Claude Code Auto-mode lesson). Co-Authored-By: Claude <noreply@anthropic.com>
When DEEPCODE_RISK_CLASSIFIER=1 and a provider is available, Session now attaches risk_classifier=build_risk_classifier(provider) to the AgentRunSpec of each real turn, so headless asks can be auto-allowed only when clearly low risk. Off by default; inert when a human approver is attached. Co-Authored-By: Claude <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Revives the risk-classifier split requested on #181 (08-17 review). The split PR #189 was opened and then self-closed during a strategy change; it was never declined in review. Rebased onto current
main(f183965), conflict-free.What
core/harness/classifier.py(289 lines; stdlib + loguru only): scores high-risk tool calls; enabled withDEEPCODE_RISK_CLASSIFIER=1(model override viaDEEPCODE_RISK_CLASSIFIER_MODEL), off by default.core/agent_runtime/runner.py: the approval path consults the classifier and fails closed (deny + log) on classifier errors, the same pattern aspermission_checker.core/events/session.py: the classifier applies to turn builds when enabled.DEEPCODE_RISK_CLASSIFIERunset there is zero behavior change.Verification
tests/test_harness_classifier.py+tests/test_classifier_wiring.py: 19 passed.f183965f59(differential-verified, unrelated).Checks
core/harness/had no classifier.