Source: assist/r5/REVIEW_REQUEST3.md
# Lead: dispatch QA1 for the frozen Round 5 LLM-agent batch Freeze SHA256: `6dd6b364ae02a550f8143354ea44b481b43d7ce3cf41b5ba347aa446bafd69c1`. The prompt/protocol/runner/checklists/sample were committed before the20 real calls. These are20 new cases, not assist001–040; exact source hashes also do not overlap prior assist/calibration cases. No new-case labels were read or used to tune the frozen agent. All outputs remain unreviewed. Please dispatch **QA1 independently**: 1. **Phase A, blind:** use `review_sample.json` and the law checklist to read the complete canonical IR sources and inventory case-specific target-element applications. Do NOT open sample.json (selection strata), REPORT.md, costs, proposals/, review_packets/, or raw/ until that inventory is saved. Selection strata are proxies, not labels. Absence of evidence is a valid finding. 2. Save `annotations/<sample_id>.json` with reviewer QA1, sample_id, source_sha256, blind_before_proposals, review_complete and a gold array. Use fully-qualified element IDs plus explicit law_id, exact Unicode `[start,end)` offsets and verbatim quotes. Distinguish current court/current parties from comparator facts, legal tests, syllabus, dissents and historical/off-target matters. 3. **Phase B:** open `review_packets/*.json`, whose IDs match full packets but omit model reasons, case-role predictions, cost and selection strata. Judge element/passage relevance and declared role/polarity separately as needed. Copy proposal IDs exactly. Use `element_accept` and `variant_accept` booleans; all variants are absent, so variant_accept=false is not a variant error. 4. Report independent proposal precision, case coverage and recall under an explicit span convention. Separately identify same-element alternate evidence rather than conflating strict containment with substantive recovery. Do not count model abstentions as proven negatives. Every retained quote passed exact mechanical checks; that is not semantic acceptance. 5. Record actually observed annotation/review times, or null if unmeasured. No causal effort-saving claim is supported without a randomized control. After annotations land, `python3 neov2/tools/score_assist.py --root neov2/assist/r5` can score the compatible judgments. Put findings in `neov2/qa/assist-r5-QA1.md`. Do not edit the frozen prompt/protocol or use these labels for retrospective model tuning. Any next proposer revision needs a new held-out batch. The developer has supplied proposals, not ground truth. This file requests Lead's dispatch; it does not assert that dispatch or independent QA occurred.