Source: assist/r6/REVIEW_REQUEST4.md
# Lead: dispatch independent QA1 + QA2 for Round 6 Frozen protocol/input hash: `bc283efcd46bbe30a220af9f911af8c0838e962e05a34000645fb4062ee062f7`. Thirty cases were selected before inference, excluding the earlier60 assist cases. The prompt, catalogue, global exclusions, recall policy and reviewer assignments were committed before any real-case calls. No new review labels have been used for tuning. Please dispatch both reviewers. ## Assignment (do not infer expected labels) - QA1 primary15: r6-001, r6-003, r6-005, r6-006, r6-009, r6-010, r6-012, r6-014, r6-017, r6-019, r6-021, r6-023, r6-025, r6-027, r6-029. - QA2 primary15: r6-002, r6-004, r6-007, r6-008, r6-011, r6-013, r6-015, r6-016, r6-018, r6-020, r6-022, r6-024, r6-026, r6-028, r6-030. - Shared6: r6-001, r6-002, r6-003, r6-004, r6-006, r6-007. - Each reviewer therefore reviews18 cases (own15 plus3 from the other half). `review_assignments.json` has the exact `review_cases` lists. Union30, overlap6. ## Phase A: independent blind inventory Use ONLY `review_sample.json`, canonical source texts and the leakage-filtered `catalogue.json` to establish ground truth. Do not open sample.json (strata/prior roles), catalogue_audit.json, REPORT/cost files, proposals/review_packets, raw traces, authored CASE files for these test IDs, or the other reviewer's work. Catalogue support excludes every test ID/source alias across all statutes. Inventory ALL actual applications by this court of target elements to these parties/counts. Distinguish historical facts, comparator cases, off-target laws, syllabi, pure rules, assumed/undisputed points and noncontrolling opinions. Mark uncertainty and incomplete reading honestly. The provided IR can be capped; completeness means the provided source, not unknown missing official text. For each gold item record fully-qualified element_id, law_id, exact quote and Unicode `[start,end)` offsets, role/polarity and party/count. Independently map an existing eligible variant or describe a materially new factual pattern. Do not copy a proposed NEW label into Phase A. Empty inventories are valid. Write to separate directories `annotations-QA1/` and `annotations-QA2/`, named `<sample_id>.json`, including reviewer exactly QA1/QA2, sample_id, source_sha256, blind_before_proposals, review_complete, gold, and actually measured times (or null). Do not compare shared-case inventories until both are saved. ## Phase B: adjudicate frozen candidates Then open the assigned `review_packets/*.json`. First judge element/quote/role against the actual source, then variant fit and the supplied variant justification. Existing membership alone is not fit. NEW is relative to the eligible catalogue; check whether it is a useful generalizable distinction, a synonym of an existing label, over-specific, unsupported, or attached to the wrong element. NEW labels are proposals, never automatic catalogue additions. Preserve uncertainties. Return judgments with exact proposal_id, element_accept and variant_accept booleans and reasons. For compatibility, variant_accept is JOINT correctness: true only if the element application AND the variant/NEW justification are valid. Optional variant_fit_alone may be recorded separately. Do not mark disagreement as consensus. Null/invalid judgments are missing data, not presumed rejection. Report each reviewer's precision/recall and existing/NEW acceptance separately. Primary15+15 judgments form the non-duplicated30-case result; cross-reviews serve agreement, not extra independent cases. Distinguish strict-span recall from same-element alternate-evidence recovery and from missed applications of an already-covered element. No adjudication before raw agreement is recorded. Shared-case agreement should cover gold application presence, element_accept and variant_accept, with raw counts/raw agreement/Cohen kappa (undefined where appropriate). Six shared cases are small; give denominators and disagreements. No causal human effort-saving claim follows sequential Phase A/B times. ```bash python3 neov2/tools/review_r6.py --root neov2/assist/r6 ``` The helper validates assignments/reviewer IDs/source pins, preserves multiple proposals per element and reports pending/null rather than inventing reviews. Reports may be `neov2/qa/assist-r6-QA1.md` and `assist-r6-QA2.md`. The developer has made no legal acceptance or held-out recall/novelty claim.