Source: assist/r4/DESIGN.md
# Frozen Round 4 proposer design Scope: develop only on assist-001..020. After this design, implementation, configuration, catalogue, dependencies and dev evidence are hashed, generate assist-021..040 once. No held-out source, old held-out proposals, or held-out annotations were opened during this round's development. Round 3 had already used these cases; this is a new frozen proposer experiment, not never-seen data. ## Fixed pipeline 1. Condition on the pilot law supplied by the citation-based sample manifest. This is assisted extraction for a known cited law, not statute discovery. Only that law's declared elements may be emitted; cross-law output is zero by construction, not evidence of perfect relevance classification. 2. Require a target statute citation or discussion anchor (e.g. RICO, Sherman, antitrust, Title VII; exact patterns in config.json). Candidate windows must lie within 3,500 characters of an anchor. Citation alone is insufficient. 3. Abstain when the first 4,000 characters frame a selected collateral or procedural issue. Remove candidate windows with procedural/comparator cues or explicit syllabus/headnote markers. These are lexical heuristics, not an adjudicated legal role classifier. Mixed merits/procedural cases can be wrongly excluded; generic rule passages can still survive. 4. Make overlapping windows of 1–3 sentence-aligned approximately 55-word segments, each 100–1,200 characters. Exact source offsets/text are preserved. These are bounded review contexts, not claimed minimal sufficient facts. 5. Build an element-conditioned MiniLM query from the frozen LAW element name, verbatim statutory text (up to 600 query characters), and descriptions of known LAW variants (up to 300 query characters). Query truncation affects synthetic retrieval queries only, never the source quotations. 6. Require an element-specific lexical stem/phrase plus application-like party/complaint/court cues within a 250-character context margin. Skip top-level x-* context elements and elements marked exception/defense. This narrows scope and can omit useful defenses; it is an explicit recall limitation, especially for the unseen development domains. 7. Score with fixed cosine >=0.29, lexical overlap bonus0.015 (capped4), application bonus0.12, primary-element bonus0.06, anchor-distance penalty0.04 and length penalty0.04. Choose one window per element, at most five overall. No minimum quota: empty packets are allowed. Tie ordering is deterministic. 8. Emit no exact variant assignment (`variant_id:null` plus abstention reason). Existing CASE-specific variant labels are not transferable automatically. This withdraws unsupported specificity; variant precision is undefined, not improved to 100%. Known variants still inform synthetic queries. All parameters, lexical cues and stop words are in config.json or propose_r4.py. The real pinned ONNX MiniLM and text/model-keyed embedding cache are reused. No cross-encoder or new package was added. Law source fingerprints are retained in catalogue.json; only that frozen catalogue is consumed at generation time. ## Development selection and measurements Three development iterations are preserved: dev_trial1.json (law conditioning, anchors, initial role filter), dev_trial2.json (opening/context role gates and element-specific cues), and dev_metrics.json (overlapping third segment and removal of an exact-word gate that incorrectly rejected morphological matches). The final configuration was selected using only these 20 cases and QA1's labels. There are no hardcoded case names, IDs, gold phrases or source offsets in the proposer. Hand-selected lexical cues nevertheless reflect development errors. Final development output: 66 proposals, versus 100 originally. On the nine empty-gold cases in QA1 JSON, proposals fall from45 to13, with six empty packets. All cross-law output is eliminated, and median span length falls from1,650 to860 characters (maximum1,198). Strict same-element gold-span coverage rises only from1/19 to2/19. This is modest development retrieval coverage, not independently measured precision or a reliable extraction solution. New packets have not been independently judged. Gold-containing-proposal rate is not a precision estimate: other acceptable evidence may not share the annotator's chosen span. QA1 reported12/100 precision and2/19 recall; those independent report values remain unchanged. The JSON contains100 judgment records, but one references an unknown proposal ID, leaving99 recognized judgments. It contains9 empty-gold cases rather than the report's11. `evaluate_r4.py` normalizes short gold element IDs using their explicit law ID and declares full-containment and50%-overlap conventions; baseline full containment is1/19 under that convention. We neither repair gold evidence nor relabel proposals to force agreement with the report. Some gold items are legal-standard passages, so a facts-oriented role filter can conflict with this benchmark. See ERROR_ANALYSIS.md for source checks. The split is statute-ordered: development contains8 Sherman,8 RICO and4 drug cases (the four drug cases are empty-gold); held-out contains4 drug,8 §1983 and8 Title VII cases. No §1983/Title VII development labels were available. This is substantial domain shift, not a random IID split. Held-out precision, recall, variant quality and effort savings must remain unknown pending QA1 review. ## Freeze contract `propose_r4.py freeze` refuses an existing freeze or existing held-out packets. It writes freeze.json and its SHA256 in freeze.sha256. The manifest hashes the implementation, configuration, catalogue, relevant runtime Python sources, model identity, sample metadata, this design, development metrics/error analysis and all20 original QA1 development annotations. The manifest is committed before held-out generation. Held-out generation validates hashes before any source read and before each case; modified inputs, missing freezes and packet overwrites fail closed. Every held-out packet records the freeze hash and generation time. Reproduction uses the existing search venv and model cache: ```bash # Development only, before a freeze exists: /tmp/neov2-search-venv/bin/python neov2/tools/propose_r4.py catalogue /tmp/neov2-search-venv/bin/python neov2/tools/propose_r4.py dev python3 neov2/tools/evaluate_r4.py # Close development; commit the freeze before generating held-out packets: /tmp/neov2-search-venv/bin/python neov2/tools/propose_r4.py freeze /tmp/neov2-search-venv/bin/python neov2/tools/propose_r4.py heldout ``` A published frozen directory is immutable; repeat experiments in a separate checkout/output scope rather than deleting the freeze or overwriting packets. Independent QA1 review is requested separately. Source corpus files and original Round 3 proposals/QA labels remain unchanged.