Source: qa/presentation-review-Fable.md
# Presentation PoC review — Fable (independent; I did not author the presentation)
Reviewed: live https://v2.neochart.com/explore/ against the builder's snapshot
(`/home/mma/worktrees/neov2-r8/neov2/db/neov2.sqlite`, logical digest `b8882353…318f8`, commit `fef06f6`,
not yet on master) and the read-only corpus. The live pages I fetched are byte-identical to the local build.
Check script: `neov2/qa/presentation-review-Fable.py` (re-slices every quote from the corpus and recomputes
counts with its own SQL; it does not import the builder). Conflict of interest to declare: 105 of the 165
case files behind this snapshot are mine, so part 2 criticises my own labels as much as the pages.
## Verdict
* **Fidelity: PASS.** No quotation, offset, hash or count on the three laws' pages is wrong. Non-merits
blocks are not counted. No script, no unescaped source text, links resolve.
* **Truthfulness of status: PASS on what it says, FAIL on what a reader needs.** "Legal review status:
unavailable" is true and uniform; it does not let anyone tell a QA-passed case from an unreviewed one,
and the snapshot is silently 135 case files behind master.
* **Usefulness for the operator's goal: view A is a usable reference; view B, as built, is not.** The matrix is
a 93 %-empty diagonal of 1s because its columns are case-specific labels. Keep A, rebuild B on routes.
## 1. Fidelity
### 1.1 Quotations (42usc1983, 18usc1962, 15usc1)
Full sweep, not only a sample: every `blockquote[data-quote-id]` on the law, element and case pages of the
three laws — 1 819 quotations on 143 pages (963 distinct ids: statute text, element anchors, interpretive
rules, case evidence). For each: corpus record loaded independently → `text[start:end]` equals the
HTML-unescaped DOM text equals the DB row's quote; DB start/end/source equal the page's; SHA-256 of the
full source text equals the page's. **0 mismatches.** Ten per law listed here (seeded sample):
**42usc1983 — 10 quotations**
| # | kind | quote id | page | offsets | corpus slice = DOM text = DB row | source SHA-256 |
|---|---|---|---|---|---|---|
| 1 | element text | `42usc1983:x-malicious-prosecution` | element | 306:402 (96 chars) | match | match |
| 2 | element text | `42usc1983:damages` | element | 404:516 (112 chars) | match | match |
| 3 | interpretive rule | `42usc1983:deprivation.procedural-due-process:rule:1` | element | 30072:30393 (321 chars) | match | match |
| 4 | interpretive rule | `42usc1983:person:rule:0` | element | 7140:7341 (201 chars) | match | match |
| 5 | interpretive rule | `42usc1983:deprivation.procedural-due-process:rule:0` | case | 27361:27504 (143 chars) | match | match |
| 6 | case evidence | `42usc1983:/f-supp/567/1159-01:0:1` | case | 5010:5196 (186 chars) | match | match |
| 7 | case evidence | `42usc1983:/f-supp/772/0583-01:0:0` | element | 4117:4213 (96 chars) | match | match |
| 8 | case evidence | `42usc1983:/us/502/0021-01:4:0` | element | 15323:15472 (149 chars) | match | match |
| 9 | case evidence | `42usc1983:/us/503/0115-01:0:3` | element | 13393:13588 (195 chars) | match | match |
| 10 | case evidence | `42usc1983:/f-supp/567/1159-01:8:0` | case | 26985:27219 (234 chars) | match | match |
**18usc1962 — 10 quotations**
| # | kind | quote id | page | offsets | corpus slice = DOM text = DB row | source SHA-256 |
|---|---|---|---|---|---|---|
| 1 | element text | `18usc1962:racketeering.bankruptcy-fraud` | element | 148:169 (21 chars) | match | match |
| 2 | element text | `18usc1962:injury` | element | 33:68 (35 chars) | match | match |
| 3 | interpretive rule | `18usc1962:enterprise:rule:3` | element | 17447:17743 (296 chars) | match | match |
| 4 | interpretive rule | `18usc1962:enterprise:rule:4` | element | 22447:22820 (373 chars) | match | match |
| 5 | interpretive rule | `18usc1962:conspiracy:rule:2` | element | 24368:24514 (146 chars) | match | match |
| 6 | case evidence | `18usc1962:/f3d/660/0346-01:3:0` | case | 11421:11491 (70 chars) | match | match |
| 7 | case evidence | `18usc1962:/f3d/660/0346-01:3:2` | element | 25542:25670 (128 chars) | match | match |
| 8 | case evidence | `18usc1962:/f-supp/567/1146-01:3:0` | case | 18960:19064 (104 chars) | match | match |
| 9 | case evidence | `18usc1962:/f-supp-2d/926/1143-01:5:0` | element | 41428:41566 (138 chars) | match | match |
| 10 | case evidence | `18usc1962:/f2d/624/1118-01:6:3` | case | 19436:19577 (141 chars) | match | match |
**15usc1 — 10 quotations**
| # | kind | quote id | page | offsets | corpus slice = DOM text = DB row | source SHA-256 |
|---|---|---|---|---|---|---|
| 1 | element text | `15usc1:agreement.x-monsanto-exclusion-of-independent-action` | element | 57:133 (76 chars) | match | match |
| 2 | element text | `15usc1:x-horizontal-cartel-via-dual-distribution` | element | 57:133 (76 chars) | match | match |
| 3 | interpretive rule | `15usc1:x-ancillary-restraints:rule:1` | element | 16295:16610 (315 chars) | match | match |
| 4 | interpretive rule | `15usc1:agreement:rule:1` | element | 17655:17960 (305 chars) | match | match |
| 5 | interpretive rule | `15usc1:restraint:rule:0` | case | 5624:5851 (227 chars) | match | match |
| 6 | case evidence | `15usc1:/f2d/589/0451-01:1:2` | case | 14720:14823 (103 chars) | match | match |
| 7 | case evidence | `15usc1:/f2d/361/0874-01:5:0` | case | 5404:5522 (118 chars) | match | match |
| 8 | case evidence | `15usc1:/f3d/496/0403-01:1:6` | case | 16472:16635 (163 chars) | match | match |
| 9 | case evidence | `15usc1:/f-supp-2d/385/0740-01:3:0` | element | 29159:29421 (262 chars) | match | match |
| 10 | case evidence | `15usc1:/f-supp/864/1175-01:0:1` | element | 6729:6859 (130 chars) | match | match |
Also checked: 7 evidence quotes in the DB contain `&`; in the DOM they are `&` and round-trip exactly.
No blockquote contains a raw `<` or a bare `&`.
### 1.2 Counts
All counts recomputed as `citation_role='applied' AND ruling_scope='merits' AND holding!='not_reached'`
(and cross-checked against the DB's own `is_instance`): the three law banners, all 80 element-page banners,
and all 350 variant rows / matrix cells with a declared variant (175 in the outlines, 175 in the matrices,
59 of them zero in each). **0 mismatches.** Ten per law:
**42usc1983 — 10 counts**
| # | count on the page | page | SQL | |
|---|---|---|---|---|
| 1 | law banner: annotation blocks | 114 | 114 | match |
| 2 | law banner: merits instance blocks | 44 | 44 | match |
| 3 | law banner: distinct cases | 12 | 12 | match |
| 4 | law banner: excluded blocks | 70 | 70 | match |
| 5 | element page `x-absolute-immunity`: blocks / merits / cases | 2/2/2 | 2/2/2 | match |
| 6 | element page `person`: blocks / merits / cases | 13/2/2 | 13/2/2 | match |
| 7 | element page `damages`: blocks / merits / cases | 12/0/0 | 12/0/0 | match |
| 8 | matrix cell `deprivation.due-process-property:property-interest-public-employment/constructive-discharge-without-resignation`: blocks/cases | 1/1 | 1/1 | match |
| 9 | matrix cell `color-of-law:public-official/exercise-of-official-authority`: blocks/cases | 1/1 | 1/1 | match |
| 10 | matrix cell `statute-of-limitations:john-doe-defendants/no-relation-back-for-unnamed-officers`: blocks/cases | 0/0 | 0/0 | match |
**18usc1962 — 10 counts**
| # | count on the page | page | SQL | |
|---|---|---|---|---|
| 1 | law banner: annotation blocks | 113 | 113 | match |
| 2 | law banner: merits instance blocks | 44 | 44 | match |
| 3 | law banner: distinct cases | 9 | 9 | match |
| 4 | law banner: excluded blocks | 69 | 69 | match |
| 5 | element page `racketeering.stolen-property`: blocks / merits / cases | 1/1/1 | 1/1/1 | match |
| 6 | element page `x-parol-evidence-rule`: blocks / merits / cases | 1/1/1 | 1/1/1 | match |
| 7 | element page `association`: blocks / merits / cases | 5/0/0 | 5/0/0 | match |
| 8 | matrix cell `conduct:operation-or-management/outside-auditor`: blocks/cases | 1/1 | 1/1 | match |
| 9 | matrix cell `conduct:operation-or-management/principals-of-scheme`: blocks/cases | 1/1 | 1/1 | match |
| 10 | matrix cell `person:no-distinct-enterprise/corporation-and-its-employees`: blocks/cases | 0/0 | 0/0 | match |
**15usc1 — 10 counts**
| # | count on the page | page | SQL | |
|---|---|---|---|---|
| 1 | law banner: annotation blocks | 89 | 89 | match |
| 2 | law banner: merits instance blocks | 29 | 29 | match |
| 3 | law banner: distinct cases | 11 | 11 | match |
| 4 | law banner: excluded blocks | 60 | 60 | match |
| 5 | element page `injury.amount`: blocks / merits / cases | 1/1/1 | 1/1/1 | match |
| 6 | element page `agreement`: blocks / merits / cases | 22/17/9 | 22/17/9 | match |
| 7 | element page `restraint.quick-look`: blocks / merits / cases | 0/0/0 | 0/0/0 | match |
| 8 | matrix cell `agreement:capacity-to-conspire/agent-implementing-principals-decision`: blocks/cases | 1/1 | 1/1 | match |
| 9 | matrix cell `standing:efficient-enforcer/speculative-damages-from-hypothetical-open-market`: blocks/cases | 1/1 | 1/1 | match |
| 10 | matrix cell `restraint.per-se:per-se-rejected/vertical-minimum-resale-price-maintenance`: blocks/cases | 0/0 | 0/0 | match |
Zero vs dash: a declared variant with no counted instance shows `0 blocks 0 cases`; a variant not declared
for that element shows `— Not declared` (850 of 918 body cells in the Sherman matrix). Rendered and
distinguishable as the notes promise.
### 1.3 Non-merits blocks
For the 57 case pages of the three laws, (rows, rows marked `Excluded from merits counts`) equals
(DB blocks, blocks with `is_instance = 0`) on every page. Excluded rows keep their holding, role/scope and
quotes visible (e.g. Dye's arguendo qualified immunity: `satisfied … applied / alternative`, excluded;
Kushner's rule-announced `association`: excluded). The largest excluded groups in the snapshot are
applied-but-not-reached (203), rule-announced (79), background (25), comparator (20), procedural (70),
lower-court (16), alternative (14). None is counted anywhere I looked.
### 1.4 Links, escaping, traversal
* 6 944 internal `/explore/…` hrefs across the 568 pages: every target exists. 215 external targets, all
`https://www.neochart.com/catalog/cases/…`; a random 12 return 200. No other host.
* 0 `<script>`, 0 inline `on…=` attributes. Element and case URLs are SHA-256 of the id, so ids cannot
shape a path; law ids are percent-encoded with `.`/`..` guarded. No id in the DB contains `< > " & \` or `..`.
* Live traversal probes (`/explore/law/..%2f..%2findex.html`, `…/18usc1962/../../../index.html`, a bogus
case id) return 403. Minor: a missing page is an S3 `403 application/xml`, not a 404 page; and there are no
`Content-Security-Policy` / `X-Content-Type-Options` headers (low risk with no JS, cheap to add at CloudFront).
### 1.5 Does the banner tell the truth?
* It says, on every page, "Legal review status: unavailable to this view." That is accurate: the DB has no
per-case review field, and the builder refuses to infer one. It also says quote verification is not
legal review. Good.
* **Flash-authored cases are not marked — because none are shown.** The snapshot was loaded at `c853444`
with 165 case files; master now has 300. The 135 newer files (the Flash extractions among them) are simply
absent, and no page says "165 of 300 case files as of <commit/date>". The digest identifies the snapshot
but not its age or what it omits. When the DB is rebuilt, Flash cases will appear with exactly the same
banner as QA-passed ones: **there is no author or review marking at all**, so the answer to "are they
marked?" is no, and will stay no until provenance is in the DB.
* The same gap hides useful positive information: 42 pilot cases were sampled by QA1/QA2 with recorded
PASS/FIX verdicts, the round-5 audit touched 23 wave-2 files, and none of that is visible.
* Data the DSL has and the pages drop: the ten v1.1 files' `?` uncertainty lines, their nine `[neutral]`
context facts (the loader's v1 view omits them), `decider`/`review`, and every file's `misfit` lines.
A reader cannot see that an annotator flagged doubt about the very block being displayed.
## 2. Usefulness for "for each element, what variants do we have; the case is an instance of the law"
Compared on 18usc1962 and 42usc1983 (15usc1 for the numbers).
### What a lawyer learns in 60 seconds
**View A (outline).** You learn the element list with a one-sentence doctrinal gloss each, which elements have
any decided case at all (RICO 18 of 30, § 1983 20 of 32, Sherman 8 of 18), and — scrolling to the bottom —
the claim formulas and the cross-element constraints. For RICO you can find `enterprise` and see twelve
labels with one case each. This is a real, readable reference page. What you do *not* learn in a minute:
which way any of those cases went, what the cases are (they are shown as `/f3d/80/1293-01`), or what the
labels mean.
**View B (matrix).** You learn almost nothing. Sherman: 18 rows × 51 variant columns, 850 "Not declared",
39 zeros, 29 ones. Every non-zero cell in the three laws is 1 except a single 2 (RICO). A matrix earns its
width when columns recur across rows or cells vary; here each column belongs to one element and nearly
each label to one case, so the table is a diagonal you must scroll sideways to read. The notes admit
"sparse and wide"; it is worse than that — the 2-D layout adds no information over A's per-element list.
### What is misleading
1. **Species labels presented as categories.** Column and row headers such as
`association-in-fact/passive-corporation-named-as-member` are one annotator's description of one case.
`neov2/variants/REPORT.md` measured it: 512 of 520 labels occur in a single case and two annotators
agree on the species 2–17 times in 60; only the *route* (the doctrinal sub-question, κ ≈ 0.97) is a
category. Both views use the folksonomy as their axis, so "what variants do we have" is answered with
a list of case nicknames. (These are largely my labels; the criticism stands.)
2. **Counts of 1 look like data.** "1 merits instance block; 1 distinct case" in a table with a count
column and heat shading invites reading frequency into n = 1. The disclaimer is in a collapsed
`<details>`. In the three laws 115 of 116 used variants have exactly one case.
3. **No holding direction.** A counted block may be `satisfied` or `not`. RICO's 44 merits instances are
15 satisfied and 29 not; § 1983 23/21; Sherman 14/15. The variant tables and matrix cells merge them, so
"enterprise — 9 variants with a case" reads as nine ways to plead an enterprise when five are ways it failed.
4. **Zero is ambiguous.** 276 of the 620 declared variants show `0 … No counted example cases`. For 100 that
is true (declared on the law side, never used). For **176 a case does use the variant, in an excluded
block** — e.g. `association-in-fact/informal-crew` is Boyle, excluded because the Supreme Court only
approved the jury instruction. The row says "no example"; the truthful row is "0 counted; 1 non-merits
use (rule announced): Boyle". This is the most concrete way the page currently hides the corpus's best cases.
5. **Cases have no names.** Titles and links are corpus paths (`/us/533/0158-01`). The loader does not
carry the caption, so a reader cannot recognise Cedric Kushner, Boyle or Monell-type authorities, and
court/date appear only on the case page, never next to an example.
6. **Outline order is by element id**, not by the law's claim structure: § 1983 opens with "Personal
involvement / causal connection", then "under color of", with claims and constraints at the very bottom.
Some names are internal ("Damages remedy (legacy CASE id; not a …)"), and markdown asterisks in glosses
show literally.
7. **The banner counts are per page but the status line is global boilerplate**; see §1.5.
### What is missing
* Route definitions (the catalogue with include/exclude rules exists in `neov2/variants/`); without them a
label is unexplained.
* Case caption, court, year and holding next to every example.
* The statute text. Element pages quote the 2–15 words each element is anchored to, three separate times
for RICO's "any enterprise"; nowhere can you read § 1962(c) with those anchors highlighted, which is the
one view that would show "the law has elements" at a glance.
* The law-side structure added in round 4 (`mode restraint`, `protocol mcdonnell-douglas`, `kind vehicle`,
defense `effect`) — the snapshot's schema has claims and constraints only.
* Any way to start from a case and see it *as an instance*: the case page is a five-column table of
blocks; it does not lay the case against the law's element list (met / not met / not reached / not addressed).
* Snapshot age and coverage (how many case files exist vs are shown).
## 3. Recommended direction
**Keep A as the spine. Drop B in its current form. Add a case-as-instance view.** The fidelity machinery
(source re-slicing, fail-closed build, digest, no JS) is the valuable part and should be kept unchanged.
Five changes, ranked by value to the operator's question, written so the builder can implement them:
1. **Make the variant axis the route, and show the pattern as text.**
Input: `neov2/variants/data/crosswalk.tsv` (raw label → route) and `CATALOGUE.md` (definitions). Loader:
add `route(id, element_id, name, definition, status ∈ established|provisional)` and `variant.route_id`.
Element section in A becomes: one row per **route** with its one-sentence definition, then counts split
by direction — `satisfied n · not satisfied n · non-merits n` — then the cases, each with its free-text
pattern (today's species) in plain type. Routes resting on one case are labelled "provisional (1 case)".
Acceptance: no page uses a species string as a row/column header; every route row shows a definition or
the words "no definition yet".
2. **Give every case a human identity and every example a direction.**
Loader: carry the `.case` header `name` and `cite` into `case(name, cite)`. Everywhere a case appears,
render `Boyle v. United States, 556 U.S. 938 (2009) — S.Ct. — satisfied [instruction approved; not counted]`
instead of the path. Acceptance: no visible link text begins with `/`; each example shows court, year,
holding and basis.
3. **Truthful coverage and review status.**
(a) Banner line: `Snapshot <digest8> built from commit <sha> on <date>: <n> of <N> case files in the
repository` (N from the build's git tree). (b) A reviewed-by mapping file, e.g.
`neov2/qa/review-status.tsv` with `case_path, law, author, reviewer, verdict, qa_file`, generated from
the QA reports; loader stores it; case and element pages show per case `author: … · QA: PASS (QA2, file)`
or `not independently reviewed`. A law banner then says "12 cases: 7 reviewed, 5 not". (c) Zero rows
distinguish "never used" from "used only in non-merits blocks (k)" and list those cases. (d) Show `?`
uncertainty lines and `misfit` lines on the case page (requires the loader to read the native v1.1 view).
Acceptance: a Flash-authored unreviewed case is visibly marked on every page where it appears.
4. **Replace B with a case × element grid ("the case is an instance of the law").**
Rows = cases of the law (caption, court, year), columns = the law's top-level elements in claim order,
cell = holding glyph and basis (`✓ proved`, `✗ pleaded`, `○ not reached`, `· not addressed`, non-merits
in a muted style), linking to the block. 12 cases × ~10 elements is dense, readable without horizontal
scroll on desktop, and answers at a glance which elements courts actually fight over. The same grid,
one row, heads each case page. Acceptance: for RICO the grid fits 1365 px without scrolling; cell
contents equal the DB holding/basis; non-merits cells are visually distinct and uncounted.
5. **Put the statute text next to the outline, and order the outline by the claim.**
Render the section text once at the top of A with each element's anchor span highlighted and linked
to its outline entry (anchors and offsets are already verified); order top-level elements by first
appearance in the primary claim expression, show the claim formula and constraints *above* the tree, and
add the law-side `mode` / `protocol` / `vehicle` / `effect` blocks when the snapshot includes them.
Acceptance: every highlighted span equals `text[start:end]`; the first element shown is the first operand
of the first claim.
Smaller fixes: move the n-is-tiny caveat out of `<details>` into one visible sentence; render gloss
markdown or strip it; serve a 404 page; add CSP and `nosniff` headers.
## 4. Weaknesses of this review
* Fidelity was checked exhaustively only for the three named laws (1 819 quotes, 433 counts); the other
twelve laws were covered only by the site-wide link, script and escaping scans.
* No browser or mobile rendering check: I read HTML and fetched pages with curl. Layout claims in the
author's notes (390 px behaviour, tables scrolling) are untested by me.
* Legal correctness of what the pages display was not re-reviewed here; that is the cases' QA, and I am
the author of most of them.
* The usefulness judgments are one reader's; no lawyer or operator was observed using the pages. The
60-second claims are my own reading, not a user test.
* I compared the live site with the builder's local output and found them identical for the pages
fetched (7 law/index pages plus the sampled quote pages), not for all 568.