qa/assist-r5-QA2.md

Source: qa/assist-r5-QA2.md
Exact source text

# QA2 — independent review of R5 whole-opinion proposals

**Audited tree:** `523f4a11baad1f0d092243fecfc68f1f87855d6a` (`origin/master`). I read the 37 candidate passages and their full local contexts before opening QA1 annotations/report. I also fixed an eight-case source-inventory sample before opening QA1: four cases with proposals and four with no retained proposals. The inventory includes full canonical source hashes and unique exact offsets.

## Proposal-by-proposal comparison

Each retained quote matched its canonical IR source hash and `[start,end)` slice; each ID existed in the target-law checklist. All model role fields were `applied`. My blind first pass accepted all 37 passages as current-case applications of their named target elements. QA1 accepted 36 and rejected one:

| Sample | Proposal ID | Target element | QA2 blind | QA1 |
|---|---|---|---:|---:|
| r5-002 | `r5-002-e984f30af0e5` | `42usc2000e-2:hostile-environment` | PASS | PASS |
| r5-002 | `r5-002-1ceabe18740b` | `42usc2000e-2:adverse-action` | PASS | PASS |
| r5-002 | `r5-002-714744cf761a` | `42usc2000e-2:retaliation` | PASS | PASS |
| r5-002 | `r5-002-598b09ed7392` | `42usc2000e-2:retaliation` | PASS | PASS |
| r5-002 | `r5-002-c0ba84f1c6e6` | `42usc2000e-2:because-of` | PASS | PASS |
| r5-003 | `r5-003-a2a813559bbf` | `42usc2000e-2:adverse-action` | PASS | PASS |
| r5-003 | `r5-003-8dc57ad57e07` | `42usc2000e-2:because-of` | PASS | PASS |
| r5-004 | `r5-004-02efe0f40713` | `15usc1:restraint` | PASS | PASS |
| r5-004 | `r5-004-f97eb954dfa2` | `15usc1:agreement` | PASS | PASS |
| r5-004 | `r5-004-9c03e151949d` | `15usc1:restraint.per-se` | PASS | PASS |
| r5-004 | `r5-004-73ffd60f62eb` | `15usc1:restraint` | PASS | PASS |
| r5-006 | `r5-006-05f3d17878ee` | `18usc1962:racketeering.mail-fraud` | PASS | FIX |
| r5-007 | `r5-007-a04d9e11963e` | `18usc1962:pattern.continuity` | PASS | PASS |
| r5-007 | `r5-007-bc0c5102c200` | `18usc1962:racketeering.securities-fraud` | PASS | PASS |
| r5-008 | `r5-008-e459ef82bf32` | `42usc2000e-2:impact.alternative` | PASS | PASS |
| r5-008 | `r5-008-fd947b940b5d` | `42usc2000e-2:because-of` | PASS | PASS |
| r5-008 | `r5-008-d3a09f33b984` | `42usc2000e-2:impact` | PASS | PASS |
| r5-008 | `r5-008-18ac30f2e385` | `42usc2000e-2:impact.business-necessity` | PASS | PASS |
| r5-009 | `r5-009-3a7aa04d9679` | `15usc1:statute-of-limitations` | PASS | PASS |
| r5-010 | `r5-010-33042ee9fc32` | `15usc1:restraint` | PASS | PASS |
| r5-010 | `r5-010-1b91f5461ca3` | `15usc1:agreement` | PASS | PASS |
| r5-011 | `r5-011-9cf19b92cec9` | `42usc2000e-2:retaliation` | PASS | PASS |
| r5-011 | `r5-011-97f9a6796ed3` | `42usc2000e-2:employer-reason` | PASS | PASS |
| r5-011 | `r5-011-bc0bc63a5951` | `42usc2000e-2:because-of` | PASS | PASS |
| r5-011 | `r5-011-d20b72f2d9d0` | `42usc2000e-2:because-of` | PASS | PASS |
| r5-011 | `r5-011-ee7bd0197c42` | `42usc2000e-2:pretext` | PASS | PASS |
| r5-012 | `r5-012-b26a83a433ad` | `15usc1:state-action-immunity` | PASS | PASS |
| r5-012 | `r5-012-3879ff588cbc` | `15usc1:restraint.rule-of-reason` | PASS | PASS |
| r5-012 | `r5-012-68d8fcdc8aba` | `15usc1:injury` | PASS | PASS |
| r5-012 | `r5-012-6eff6b5e9374` | `15usc1:agreement` | PASS | PASS |
| r5-016 | `r5-016-ab5b00cbc7c7` | `42usc1983:qualified-immunity` | PASS | PASS |
| r5-016 | `r5-016-ff68a04e4e88` | `42usc1983:statute-of-limitations` | PASS | PASS |
| r5-016 | `r5-016-09a3b9c8fb3d` | `42usc1983:policy` | PASS | PASS |
| r5-018 | `r5-018-af3a10529ec1` | `42usc1983:statute-of-limitations` | PASS | PASS |
| r5-019 | `r5-019-b4add22c2b55` | `42usc1983:qualified-immunity` | PASS | PASS |
| r5-019 | `r5-019-6f988c9c5f60` | `42usc1983:deprivation.fourth-amendment` | PASS | PASS |
| r5-019 | `r5-019-c0e2d7be538f` | `42usc1983:deprivation.fourth-amendment` | PASS | PASS |

The independent first-pass agreement is **36/37 (97.3%)**. The one difference is `r5-006-05f3d17878ee`. I initially accepted it because the present court states that defendants did not contest the complaint's alleged mail-fraud predicates and denies that particularity argument. On reconsideration under the stricter requirement that the passage show the court applying the target element, I agree with QA1's rejection: the quoted sentence records an uncontested pleading concession and cites *GEICO v. Korn*; the opinion does not independently evaluate mail-fraud elements. I mark it **FIX** in the final judgment. This post-report reconsideration is disclosed and is not counted as independent agreement. The initial all-positive QA2 marginal makes Cohen's κ 0 despite 97.3% raw agreement; κ is uninformative here because QA2 assigned no negative labels in the blind first pass.

All other proposals pass the strict relevance check. The current court applies them to the case's specific claims, evidence, or procedural defenses: employment discrimination applications in r5-002/003/008/011; Sherman Act agreement, tying, limitations, immunity, effects, and injury in r5-004/009/010/012; RICO continuity and securities-fraud bar in r5-007; and §1983 qualified-immunity, limitations, municipal-policy, deprivation, and relation-back determinations in r5-016/018/019. In r5-009 the court applies the criminal §1 limitations defense to alleged overt acts. In r5-016/018, limitations rulings are current §1983 procedural applications, not merits findings. In r5-019, the two Fourth Amendment passages address distinct current arrest and force applications. No proposal uses a dissent, headnote, generic legal test, or non-target statute as an applied element.

## Blind eight-case gold inventory

Inventory items are target-element applications in the complete opinion, not abstract rules or prior-offense history. For strict span recall, a proposal counts only when its quote contains the entire inventory span for the same target element in the same case. Same-element evidence elsewhere is reported separately.

| Sample | Target law | Blind inventory before QA1 | Exact proposal spans contained |
|---|---|---|---:|
| r5-001 — *Trump v. Illinois* | `42usc1983` | None. The stay application is under 10 U.S.C. §12406; the opinion expressly does not decide whether any immigration encounter violated constitutional principles. | 0 |
| r5-002 — *Martinelli v. Bancroft Chophouse* | `42usc2000e-2` | Hostile environment `[10318,10572)`; quid-pro-quo/because-of `[12470,12674)`; retaliation `[16172,16467)`; constructive-discharge/adverse-action `[17410,17564)`. | 2/4 |
| r5-004 — *Wholesale Alliance v. Express Scripts* | `15usc1` | Per-se horizontal concerted action `[15801,15887)`; rule-of-reason concerted action `[21131,21260)`; tying/conditioning `[23500,23607)`; tying seller's direct economic interest `[28166,28287)`. | 4/4 |
| r5-005 — *United States v. Canalichio* | `18usc1962` | None. §1962(d) is historical conviction context; the §2255 opinion adjudicates ineffective-assistance claims, not RICO elements. | 0 |
| r5-006 — *GEICO v. Quality Diagnostic* | `18usc1962` | **Initial inventory:** mail-fraud predicate pleading concession `[10344,10573)`. **Strict re-review:** exclude; the court reports that defendants did not contest the allegation and cites another court, without independently applying the predicate. | 0/0 final; 1 provisional span would have been contained |
| r5-008 — *Freyd v. University of Oregon* | `42usc2000e-2` | Title VII comparator/cause `[27626,27744)`; disparate-impact statistics `[32505,32758)`; business necessity `[34239,34345)`; alternative practice `[36635,36847)`. | 4/4 |
| r5-013 — *United States v. Charles* | `21usc841` | None. The appeal reviews the concurrent-sentence doctrine and a First Step Act remand; §841(a)(1) identifies a historical conviction but its elements are not adjudicated. | 0 |
| r5-015 — *Jaludi v. Citigroup* | `18usc1962` | None. The RICO claim is held arbitrable; the appeal decides which arbitration agreement governs, not §1962 elements. | 0 |

Canonical source identities (all SHA-256 values matched `review_sample.json`):

| Sample | IR record | SHA-256 |
|---|---|---|
| r5-001 | `clus_cl.jsonl#/cl/10766642` | `e8b342da9253faa1470e20b9c3a32a84eafa64c55181488ca1c692612cc33a42` |
| r5-002 | `f_supp_3d_357.jsonl#/f-supp-3d/357/0095-01` | `e341ff3eb37061e227f597bc3535d6b9e36a2d9af8bbde04a17f6618a4b5caf3` |
| r5-004 | `f_supp_3d_366.jsonl#/f-supp-3d/366/1069-01` | `203fd2ea4bd5aba48602f76e1b18bcb009a263bcd0b2a76dff64e025deb775fc` |
| r5-005 | `f_supp_3d_369.jsonl#/f-supp-3d/369/0625-01` | `8e83e4a16718432cb73952d743d73ed0b0afd00281ac508833bba25910c5afc0` |
| r5-006 | `f_supp_3d_369.jsonl#/f-supp-3d/369/1292-01` | `e8791fda9dc988366b511ca8f1b717afd166978f12237d626ba307acda98f399` |
| r5-008 | `f_supp_3d_384.jsonl#/f-supp-3d/384/1284-01` | `f135b9e4a8be7f96b5de83c7133d1d7369fcfeebc7bc8b6b1d7519a53fd85fba` |
| r5-013 | `f3d_932.jsonl#/f3d/932/0153-01` | `d2a2395a47c54946e419546c118c2dd8e9f418e9d07d7918a9d5c04fb5db47bd` |
| r5-015 | `f3d_933.jsonl#/f3d/933/0246-01` | `d8981724171b98ba706f6d3c27593a88e12a7dec181ceb75164a693ea1f87ae8` |

The inventory evidence includes r5-002's “failed to demonstrate” severity finding, the lack-of-connection conclusion for retaliation, and its constructive-discharge conclusion; r5-004's findings of no horizontal agreement, no plausible ROR collusion, no explicit tying condition, and no direct economic interest; and r5-008's insufficient-disparity finding, business-necessity finding, and failure to identify an alternative. The exact spans and end-exclusive offsets are listed above. The provisional r5-006 span is the uncontested concession quoted in the disagreement analysis.

The pre-label inventory contained 13 evidence items and 11 were strictly contained (84.6%). Excluding the r5-006 concession after the strict reread leaves 12 items, **10/12 strict span recall (83.3%)**. The two misses are in r5-002: proposals contain different passages applying the same `because-of` and `retaliation` elements, so same-element alternate evidence recovers both; same-element coverage for this small sample is 12/12. This sample is proposal-enriched and should not be treated as a prevalence estimate.

**Sampling deviation:** The pre-label sample was four proposal-bearing cases plus four zero-proposal cases. After seeing QA1's labels, those eight contain only three QA1-positive cases (r5-002, r5-004, r5-008); r5-006, selected as the fourth proposal-bearing case, has no QA1 gold. I did not replace it after seeing labels, because that would compromise the blind sample selection. Thus the requested exact four-QA1-positive/four-negative composition was not achieved. QA1's inventory for these eight contains seven gold items; five are exact-contained by the proposals (5/7, 71.4%), while two additional proposals provide same-element alternate evidence. This QA1-based comparison is separate from my own pre-label inventory above.

## Batch-level metrics and interpretation

- **Precision:** QA1's 36/37 = **97.3%** is arithmetically correct and credible for this batch. My strict review found the same lone false positive after resolving the disclosed r5-006 disagreement. With only 37 candidates, it is not a reliable general accuracy estimate.
- **Strict span recall:** QA1's 12/23 = **52.2%** is reproducible from its annotations when a match requires same case and element and the proposal span to contain the entire gold span. It is believable under that deliberately strict convention, but it measures quote-span containment, not all useful evidence recovery.
- **Alternate evidence:** A source-span comparison of all QA1 annotations against proposals finds same-case/same-element proposals for 21/23 gold items. Twelve are exact-contained; nine are different spans for the same element; two have no same-ID proposal. Thus the 52.2% strict recall materially understates same-element evidence coverage (21/23 = 91.3%) while preserving a meaningful distinction between exact recovery and alternate evidence.
- QA1's 100% positive-case recovery means each of its 12 cases with at least one inventory item had a proposal; it does not show full within-case element recall. It is conditional on QA1's inventory and this 20-case corpus.
- Variant precision remains undefined because all 37 proposals used `variant_id: null`; abstention is not an incorrect variant.

## Overall judgment

The **97.3% precision** result is believable as a measured batch outcome: 36 strong current-case applications and one concession-based false positive. The **52.2% recall** is also believable as strict exact-span containment, but it should always be reported beside the 21/23 same-element alternate-evidence figure. Neither number supports claims about future batches or causal human-effort savings. My blind eight-case sampling plan missed the requested 4/4 QA1-label quota by one case; the deviation and its cause are recorded rather than repaired post hoc.

Verification: the pinned snapshot has 20 proposal files and 37 retained candidates; all 37 quote offsets and SHA-256 source hashes matched the canonical IR and all target element IDs resolved in `checklists.json`. No proposal, annotation, source, or frozen prompt file was edited. This QA report is the only file intended for commit.