STEP 1572 — Eval Fallback Hardening (ARIS learn-in)
要旨
2026-08-29 radar で wanshuiyin/Auto-claude-code-research-in-sleep (ARIS ★15419、 Δ+28/day) 検出、 concept 比較で ARIS auto-review-loop が **cross-family reviewer 強制** (Anthropic → Codex) を default 化 と 判明 → Rei の Claude API second-opinion pipeline に **3 gap** (hardcoded model ID + silent fallback + fallback ledger 無し) を 検知、 STEP 1336 教訓 (success signal decoupled from operational state) 適用で fix。
3 fix
| Fix | Before | After |
|---|---|---|
| (a) model ID | hardcoded claude-sonnet-4-20250514 (404 not_found_error 実測) | env var INVENTION_EVAL_MODEL (default claude-sonnet-5) |
| (b) fallback surface | silent fallback (evaluatedBy=local-heuristic と 意図的 local が区別不能) | this.log warn + evaluatedBy subtype 3 種 (fallback-no-api-key / fallback-api-http-error / fallback-api-exception) |
| (c) obligations ledger | fallback 発生 record なし | data/invention/eval-fallback-log-<date>.json append (ARIS integrity-forensics ledger 軽量移植) |
Test coverage (32/32 PASS)
test/step1572-eval-fallback-hardening-test.ts — 8 section:
- useClaudeAPI=true + no API key → all tagged
fallback-no-api-key+ ledger apiKeySet=false - useClaudeAPI=false → 意図的
local-heuristic+ no ledger - HTTP 404 error →
fallback-api-http-error+ ledger reasonapi-http-404 - Network exception →
fallback-api-exception+ ledger INVENTION_EVAL_MODELenv var respected in request body + log- Default model =
claude-sonnet-5when env unset - Ledger append (2 runs same day → 2 entries)
- Corrupted ledger file → recover (fresh start、 don't crash)
Regression: STEP 237 (33/33) + STEP 1253 (30/30) clean。 STEP 1019 の 1 FAIL は pre-existing (別 tab merge 進行 副作用、 私 scope 外)。
Honest scope
- Cross-family reviewer 強制 (Gemini/GPT wire) は 本 STEP scope 外 = 別 STEP 1585 で skeleton wire (藤本さん budget 判断 待ち)
- ARIS 46 pattern integrity forensics 移植 は 本 STEP scope 外 = 別 STEP 1587 で 5-pattern minimum viable spike
- ARIS /meta-optimize 相当 (SKILL.md 自己 patch 提案) は infrastructure 先行必要 = 別 STEP 1535 で event log 基盤起草
- backward compat 完全維持 (既存
evaluatedBy: 'claude-api' | 'local-heuristic'は preserve、 追加 subtype のみ union 拡張)
Related
src/aios/daily/daily-invention-reporter.ts— 実装 file (+80 行 wire)test/step1572-eval-fallback-hardening-test.ts— test 32/32 PASS- STEP 1585 — Cross-family reviewer Gemini wire (対称 arc、 藤本さん budget 判断 blocking)
- STEP 1587 — Integrity forensics 5-pattern spike (ARIS learn-in 3 arc の 姉妹)
- STEP 1535 — Meta-optimize event log infrastructure
- Memory:
project_step1572_eval_fallback_hardening_2026-08-30.md