STEP 2226 — philosopher-debate dry2 session handoff
Handoff purpose: next session should be able to pick this up without re-reading the whole arc. Current state and next-step sequence are recorded here, in notepad, and in memory hooks.
Project context
Independent local project (藤本 伸樹's request, spec via chat-Claude(Cowork) 2026-09-23). Local repo at C:/Users/user/philosopher-debate/; not part of rei-aios source tree. Purpose: run philosophers as local-LLM bots and produce an 対立の地図 (map of disagreement), not conclusions.
Phase 1 target: hold-out predictive calibration on the Leibniz-Clarke Correspondence (1715–1716) — bots read letters 1–3, are asked to predict letters 4–5 (held out).
Pre-registration hash: 499ce29 (fixed 2026-09-24, immutable). Baseline vs debate delta is the primary evaluation.
What dry2 tested
Same practice question and same RAG index as dry1 (per pre-Step-4 check #2 discipline: index unchanged so prompt fix is the sole independent variable). Prompt changes only:
- RAG label full form:
[source: L2]→[source: Leibniz's 2nd Letter (Jan 1716), §2] - Persona system prompt gained WHO WROTE WHAT block + conditional Author-referent rule (per pre-Step-4 check #1: in Leibniz's letters "the Author" = Clarke; in Clarke's replies "the Author" = God).
- Explicit "opponent's text is what you argue AGAINST — do not adopt it as your own voice".
Result: partial pass
| Target | dry1 | dry2 | Status |
|---|---|---|---|
| F4 third-person self-attack (Clarke bot says "Mr. Clarke has misinterpreted…" about itself) | Present ("Mr. Clarke, in his First Reply, has misinterpreted the Sensorium…") | Resolved ("Mr. Leibniz, in your second letter, have you not already conceded…") | RESOLVED |
| F5 Leibniz critical stance (real Leibniz criticises Newton's sensorium) | Lost ("Sensorium is in line with Materialist view" = defends Newton) | Recovered in one utterance ([2]: "I must disagree. Mathematical Principles… are not the ultimate principles.") | PARTIAL |
| Rupture phrases from pre-registered PSR/absolute-space list | 0 hits | 2 hits ("principle of sufficient reason", "absolute space") in round 0 | NEW |
| Stance classifier accuracy (dev llama3.2:latest verifier) | 3 / 6 role match | 3 / 6 role match (same raw score, misses are demonstrably wrong on inspection) | UNRELIABLE |
| Freeze detector | Fired after round 3 | Fired after round 3 | SAME |
Remaining issues (6)
- Fake French letter headings — every dry2 utterance opens with a heading like
**Premier Reponse au Second Ecrit Anglois.**; bots think they are composing new letters, not participating in a dialogue. - Topic collapse — dry1 orbited "Sensorium of God", dry2 orbits "Mathematical Principles vs Materialism". Bots pick one topic per run and stay there instead of spanning T1–T5.
- Freeze detector still fires at round 3 — pairwise embedding cosine ≥ 0.9 in majority.
- Verifier (dev llama3.2) is unreliable for stance classification: called dry2 [1] Clarke-content "leibniz", [2] Leibniz-critique "neither". Prod llama3.1:8b needs testing.
- X1 (stance alignment) also unreliable at 3B — verifier sometimes echoes the historical-stance summary instead of judging the utterance.
- X2 (citation support) = 0 outputs: (a) bots did not emit
[source: …]citations in dry2, and (b) recorder.py's lookup expects short-form "L2" but the readable label is "Leibniz's 2nd Letter, §5" — reverse mapping needs a small fix.
Next-step sequence (recorded for the next session)
- Compute verifier accuracy on dry1 + dry2 logs without re-running (speaker role is known ground truth; classifier's role-match rate = signal for whether the verifier can be trusted at all).
- Fix the X2 label-lookup bug in
src/recorder.py(~10 lines, reverse-map readable label → chunk.source). - Grep retrieved chunks for original letter running-header content (French headings like "Premier Reponse au…"); if present, strip in index preprocessing. This IS an index change → record as such and switch the comparison baseline for anything after.
- dry3 = 7B, Nagarjuna off. Change one variable at a time.
- dry4 = 7B, Nagarjuna on. Only after dry3 shows role stability.
If step 1 shows the verifier is unreliable regardless of model at 3B, the choice between (a) keeping dev llama3.2 for dry3 or (b) using prod llama3.1:8b for the verifier in dry3 has to be made before step 4.
Held items (do not change)
rag_top_kstays at 5 (per pre-Step-4 check #2 — no confounding retrieval change).- Pre-registration hash
499ce29is immutable; the primary indicator definitions (a)–(g) are fixed. - Task 17 (main measurement, 4–6 hours wall-clock, n=3 per condition) is no-go until 7B shows role stability and verifier reliability.
File paths (for the next session to open first)
- Analysis:
philosopher-debate/docs/phase-1/dry2-analysis.md - Findings + judgment corrections:
philosopher-debate/docs/phase-1/execution-findings.md(F4/F5 + pre-Step-4 3-point verification) - Fixed pre-registration (immutable):
philosopher-debate/docs/phase-1/pre-registration.md - Corpus policy:
philosopher-debate/docs/phase-1/corpus-plan.md - Threshold calibration:
philosopher-debate/docs/phase-1/threshold-calibration.md - Exploratory indicators X1/X2 (post-run addition):
philosopher-debate/docs/phase-1/exploratory-indicators.md - Main measurement prep (sleep-prevention, --resume, L8):
philosopher-debate/docs/phase-1/main-measurement-prep.md - Skeleton code:
philosopher-debate/src/(config / ollama_client / rag / persona / nagarjuna / debate / verifier / recorder / baseline / main) - Result JSONs:
philosopher-debate/results/phase-1/dry-run-{dry1,dry2}-map.json - Corpus:
philosopher-debate/corpus/leibniz-clarke/(raw + letters + rag + held-out + french-original + README + META.md)
Honest scope
- philosopher-debate is a local repo without a git remote (2026-09-24). Push is deferred; user decides later whether to create a GitHub repo.
- Verifier judgments (including Claude's own during F5 catch) are fallible; primary-source quoting is the discipline that catches errors, and 10-sample human spot check by 藤本さん is scheduled once verifier output is trustworthy enough to inspect.
- The 5-step sequence is recorded so the next session can resume without re-reading the whole arc.