Findings

Findings from the authored DSLs, annotation notes and independent reviews.

What the DSLs represent

LAW DSL defines claims, elements and sub-elements, exceptions, defenses and cross-element constraints. Statutory text and case rule lines carry verbatim quotes with Unicode offsets; source pins identify the text version. law/LAW-DSL.md

CASE DSL v1 records one opinion–law association, citation role, ruling scope, holding and basis, typed party/count/theory/opinion qualifiers, and fact polarity. Facts, rules and held passages retain verbatim text and offsets. cases/CASE-DSL.md

An instance is a decided holding with role applied and scope merits. A citation, rule announcement, comparator, hypothetical ruling, or lower-court holding alone does not meet that definition. cases/CASE-DSL.md

LAW example

Verbatim excerpt from law/15usc1.law (starting at line 34).

element agreement : and
  name Concerted action ("contract, combination ... or conspiracy") between separate economic decisionmakers
  text @57..133 | Every contract, combination in the form of trust or otherwise, or conspiracy
  text @270..330 | make any contract or engage in any combination or conspiracy
  rule /us/560/0183-01 @15568..15724 | The key is whether the alleged “contract, combination ... , or conspiracy” is concerted action — that is, whether it joins together separate decisionmakers.
  rule /us/560/0183-01 @17655..17960 | Because the inquiry is one of competitive reality, it is not determinative that two parties to an alleged § 1 violation are legally distinct entities. Nor, however, is it determinative that two legally distinct entities have organized themselves under a single umbrella or into a structured joint venture.
  rule /us/550/0544-01 @13170..13306 | we hold that stating such a claim requires a complaint with enough factual matter (taken as true) to suggest that an agreement was made.
  rule /us/550/0544-01 @14304..14548 | Hence, when allegations of parallel conduct are set out in order to make a § 1 claim, they must be placed in a context that raises a suggestion of a preceding agreement, not merely parallel conduct that could just as well be independent action.
  rule /f3d/496/0403-01 @8505..8715 | To establish concerted action, the plaintiff must present “evidence that reasonably tends to prove that the [defendants] had a conscious commitment to a common scheme designed to achieve an unlawful objective.”
  rule /f2d/589/0451-01 @11362..11517 | It is fundamental that a single person or economic entity cannot combine or conspire in isolation: there must be concerted action by a plurality of actors.
  rule /f-supp-3d/360/0730-01 @18441..18587 | standard of Rule 9(b), and accordingly, plaintiffs can allege an antitrust conspiracy by pleading circumstantial evidence of an illegal agreement.
  not-required Formal or express written/oral contract; tacit agreement or conscious commitment to a common scheme suffices (Monsanto Co. v. Spray-Rite Service Corp., 465 U.S. 752 (1984))

CASE v1 example

Verbatim excerpt from cases/15usc1/us-550-544-bell-atlantic-v-twombly.case (starting at line 16).

element agreement : not-satisfied [pleaded]
  variant parallel-conduct/parallel-resistance-to-new-entrants
  theory  conspiracy among the ILECs to thwart competitive entry by CLECs
  fact @3545..3686 [supports] | Plaintiffs say, first, that the ILECs “engaged in parallel conduct” in their respective service areas to inhibit the growth of upstart CLECs.
  fact @3714..3965 [supports] | Their actions allegedly included making unfair agreements with the CLECs for access to ILEC networks, providing inferior connections to the networks, overcharging, and billing in ways designed to sabotage the CLECs’ relations with their own customers.
  fact @25437..25629 [cuts-against absent] | To begin with, the complaint leaves no doubt that plaintiffs rest their § 1 claim on descriptions of parallel conduct and not on any independent allegation of actual agreement among the ILECs.
  rule @13170..13306 | we hold that stating such a claim requires a complaint with enough factual matter (taken as true) to suggest that an agreement was made.
  rule @14304..14548 | Hence, when allegations of parallel conduct are set out in order to make a § 1 claim, they must be placed in a context that raises a suggestion of a preceding agreement, not merely parallel conduct that could just as well be independent action.
  held @26876..27095 | we agree with the District Court that nothing in the complaint intimates that the resistance to the upstarts was anything more than the natural, unilateral reaction of each ILEC intent on keeping its regional dominance.
  gloss  Each incumbent had its own unilateral incentive to resist entrants it was forced to subsidize, so parallel resistance does not suggest a prior agreement.

Current snapshot

15 canonical LAW files at law/*.law; 165 CASE files; 165 distinct case–law associations and 164 distinct opinion IDs. Parsed annotations contain 831 blocks, 348 merits instances, and 344 distinct (law, element, variant) labels on those instances. The canonical laws declare 318 element/exception/defense IDs. Published count inputs and SHA256 manifest.

See counts by law and every declared element, including zeros. The inventory documents the counting rules. These current totals are separate from the frozen matcher export and the historical review samples below.

What worked, and what that establishes

The original 60-case migration retained all 1,172 quote tuples unchanged. The v1 review reports zero quote changes and a 60-file, 1,172-quote check with zero errors. This is source-text fidelity evidence for that migration, not certification of legal labels or of the expanded current snapshot. cases/NOTES.md qa/cases-v1-QA1.md

The notes find the element/holding/basis/variant structure useful for opinions actually applying statutes, and fact/rule/held separation useful for annotation. v1 makes citation role, ruling scope, polarity and qualifiers explicit, so non-merits material can be excluded by syntax. The sample review checks corrections such as Boyle's rule announcement and Ricci's hypothetical blocks. cases/NOTES.md qa/cases-v1-QA1.md

Annotation QA: samples and remaining gaps

The v0 reviews returned 31 PASS and 11 FIX out of 42 reviewed files: QA1 reviewed 12 and QA2 reviewed 30. That is a 26.2% FIX rate in the reviewed sample, not an estimated population error rate. Defects included conditional standard selection recorded as satisfaction, incorrect evidentiary bases, lower-court or hypothetical holdings recorded as current merits findings, and sibling-statute or comparator labels. Accurate quotes and explanatory misfit prose did not prevent these errors. cases/NOTES.md qa/cases-QA1.md qa/cases-QA2.md

The v1 QA1 follow-up records 14/14 PASS: seven prior FIX cases and seven fresh cases in that review. This is a sampled migration audit, not a certification of every case in the current corpus. qa/cases-v1-QA1.md

A separate targeted LAW spot-check reports FIX on all three catalogues reviewed: §922(g), §78j and §623 (3/3, 100% of that selected sample), despite zero quote-check errors. QA2 challenges QA1's all-PASS conclusion on statutory scope, omitted exceptions and private purchaser/seller doctrine. This is not an estimated failure rate for all laws or CASE annotations. The reports preserve the disagreement and specific requested corrections. qa/law-wave2-QA2-spot.md qa/law-wave2-QA1.md

The author has published a Round 5 repair and re-audit report. It records implementation and verification, not independent QA2 approval; QA2 re-verification is separate; consult the report for its status. law/R5-REPAIR-SUMMARY.md law/NOTES.md

The v1 notes still identify coarse single-valued role/scope; untyped mode selection and element rejection; case blocks unable to cite law constraints; flat burden shifting; one basis per block; binary whole-line polarity without credited/source status; under-inclusive quoted-rule tags; free-text qualifier values; frozen quotes limiting new annotation; no temporal/good-law or cross-law pointer; and vehicle statutes whose substantive doctrine lies elsewhere. cases/NOTES.md

The v1 reviewer also identifies remedy-limiting defenses, party-asymmetric alternative grounds, fact-to-fact rebuttal and legislative facts as gaps. The review's open issues include structured mode selection, cross-element relations, burden-shifting stages and the substantive rights behind § 1983. qa/cases-v1-QA1.md

Matcher: frozen 60-case retrieval experiment

The reported benchmark is case-level leave-one-out on a hand-selected 60-case annotation export covering five pilot laws, with full opinions as queries. It is not a blind prospective fact-only test. QA2 reproduced the counts and checked case/hash exclusions and fold-specific lexical indexes; human selection, authored labels and unknown pretraining overlap remain limits. matcher/REPORT.md qa/matcher-QA2.md

Hybrid law top-1 is 58/60 versus BM25 54/60; both achieve 60/60 top-3. Six errors improve and two worsen. The paired exact McNemar result is p = 0.289 (0.2890625): not statistically significant at usual thresholds. This sample does not establish general improvement. matcher/REPORT.md qa/matcher-QA2.md

Hybrid element recall@5 with the gold law supplied is 39.1% = 75/192 (also 75/192 end-to-end here). 46 labels lack training support. QA2's supported-label view is 75/146 = 51.4%, with a coverage ceiling of 146/192 = 76.0%. Unsupported labels remain in the main denominator. matcher/REPORT.md qa/matcher-QA2.md

Exact variant recall@5 is 1/235 = 0.4%; 228/235 are unseen in their training fold. Among the seven supported variants it is 1/7 = 14.3%; the coverage ceiling is 7/235 = 3.0%. These rates measure recovery of authored labels, not whether a neighbor is legally analogous or the legal outcome is correct. matcher/REPORT.md qa/matcher-QA2.md

The fixed cosine ≥ 0.5 diagnostic accepted 60/60 positives and 10/10 negative proxies: zero specificity on this selected proxy set. These were citation-screened cases, not adjudicated non-applicability examples. The threshold failed to support abstention; its result does not estimate specificity across the wider case population. matcher/REPORT.md qa/matcher-QA2.md

Open questions recorded in the evidence

How should the representation express relational constraints, sequential burdens, multiple bases, party/count-specific outcomes, remedy limitations and constitutional rights behind vehicle statutes? How should opinion segmentation, temporal status and shared or sibling-law elements be represented? These are unresolved modelling issues in the annotation notes and v1 review. cases/NOTES.md qa/cases-v1-QA1.md

Can retrieval generalize beyond this curated collection, recover labels with sparse training support, and support a calibrated no-match decision on independently adjudicated negatives? The matcher report and QA2 leave those questions open. matcher/REPORT.md qa/matcher-QA2.md

The assistance protocol requires an independent blind evidence inventory before proposal adjudication, separate passage and variant judgments, and explicit unreviewed coverage. Only exhaustive blind inventories establish recall denominators. Proposed labels are not accepted gold without independent adjudication; effort savings remain unestablished without a separate randomized assisted-versus-unassisted timing study. assist/ANNOTATION_PROTOCOL.md