feat/lc-59-math #193

Merged

longjacksonle merged 2 commits from feat/lc-59-math into main 2026-05-24 19:14:28 +02:00

Conversation 0 Commits 2 Files changed 7 +1356 -3

longjacksonle commented 2026-05-24 19:13:25 +02:00

Owner

Copy link

Summary

Server-rendered LaTeX math in message bodies. Inline $x^2$ and display $$\int_0^1 f(x)\,dx$$ typeset to MathML via
pulldown-latex (pure Rust, no JS engine, no
client-side dependency, no separate stylesheet). MathML is natively accessible to
screen readers and renders via the browser's built-in math layout, fitting the existing
server-renders-HTMX-swaps architecture. Detection runs inside markdown::render_inner's
Event::Text branch, so code spans and code blocks remain literal by the same
mechanism that already protects mentions and emoji.

Decision log (load-bearing surfaces)

Dependency choice: pulldown-latex pinned to =0.7.1. Only candidate that satisfies
every hard constraint (no client JS, no runtime JS engine, no separate stylesheet,
SR-accessible via MathML, pure Rust). katex-rs (xu-cheng) requires a JS engine at
runtime and HTML+CSS output, ruling it out on two of the constraints. mathyank doesn't
exist on crates.io; the closest analogue (RaTeX) outputs Canvas/PNG/SVG (image-based,
ticket explicitly rejected). The pin is exact (=, not ^) so a routine cargo update cannot upgrade. Re-run the LC-59 spike before merging any version bump.

Macro recursion is a process-abort vector. The blocklist is the primary mitigation.
Spike-confirmed: \def\x{\x\x}\x stack-overflows the parser, and Rust's runtime
stack-overflow handler calls abort() unconditionally - uncatchable by catch_unwind,
and a bounded-stack worker thread does NOT contain it either.
views::math::blocklist_re rejects any span containing \def, \edef, \xdef,
\gdef, \let, \futurelet, \newcommand, \renewcommand, \providecommand,
\href, \url, \csname. Word-boundary regex, whitespace-tolerant. \let is
load-bearing: aliasing closes the chain. Denylist, not a proof - strong but
version-pinned.

Deep nesting is safe at any reasonable depth. Parser is iterative, not call-stack
recursive - spike-confirmed to depth 6400.

Four failure modes converge on literal-fallback: caps exceeded, blocklist match,
push_mathml returns Err, output contains <merror>. Plus a fifth: catch_unwind
around the render call catches non-stack-overflow panics. Per-render Storage +
Parser = no shared state.

Display-mode CSS. math[display="block"] { margin: 0.5rem 0; }. We do NOT set
display: block on display math: that CSS property overrides the browser's native
MathML layout and causes children to stack vertically (eyeball-discovered). The
display="block" ATTRIBUTE already makes the element a centered block via native
rendering.

Unplanned shared-behavior change: the Text-event merge

pulldown_cmark splits a single text run into multiple Event::Text events at
backslash-escape boundaries. Math detection per-event therefore could not find any span
containing a LaTeX escape. The fix merges consecutive Event::Text events into one
before the inline-rewrite filter_map runs. Safe across code-block boundaries by
construction.

The merge ALSO changes how mention / emoji / URL detection sees text at backslash
boundaries: a single contiguous string rather than two halves. Mentions: benign. URLs:
common shape works identically. See known-limitations for one edge case.

Known limitations

1. URL abutting a backslash-escape goes from one anchor to zero. Source
   https://example.com\!end becomes https://example.com!end after cmark, which
   linkify rejects. Pre-merge would have found the URL in the first half-event. Pinned by
   bare_url_abutting_backslash_escape_no_anchor_documented_limit. Not chat-realistic;
   text is preserved (not lost, not corrupted, not a security hole).
2. Math inside markdown link labels is intentionally not typeset in v1.
3. Math spanning inline-markdown boundaries ($x_*y*$) is not detected.
   Single-text-event scoped.
4. Backslash-escaped $ in chat text (\$) is not a literal-dollar escape. Users
   wanting a literal dollar should use a code span.
5. User-defined macros are blocked. Not supported in v1.

Test plan

• WebView2 hand-test (Windows). Build the desktop binary on Windows, run, paste
  the sample below into a room. Pass criterion: every typeset case visually correct.
  Specifically verify the corrected display-CSS: the display integral renders horizontal
  and centered, NOT as a vertical character-by-character stack.
• WKWebView hand-test (macOS). Same procedure if a Mac is available.
• WebKit2GTK was confirmed pre-merge via the LC-59 spike.

Sample message for the hand-test:

Inline: $x^2 + y^2 = z^2$
Sum: $\sum_{i=1}^{n} i = \frac{n(n+1)}{2}$
Integral: $\int_0^\infty e^{-x^2}\,dx = \frac{\sqrt{\pi}}{2}$
Greek: $\alpha, \beta, \gamma, \Sigma, \Omega$
Sub/sup stack: $a_i^j$
Fraction: $\frac{a + b}{c - d}$
Display: $$\int_0^1 f(x)\,dx$$
Mixed: pre $x^2$ between $\sqrt{y}$ post
With mention: $E = mc^2$ thanks @someone
Inside inline code: `$x^2$ literal`

Automated coverage in this PR: 41 math-module unit tests, 9 markdown integration tests,
4 route-level round-trip tests.

Files

File	Notes
server/Cargo.toml	pulldown-latex = "=0.7.1" exact pin
server/src/views/math.rs (new)	Scanner, render boundary, blocklist, caps,
catch_unwind, <merror> detection, 41 tests
server/src/views/mod.rs	pub mod math;
server/src/views/markdown.rs	Text-event merge, Event::Text wiring, 9 new tests
server/assets/main.css	Two math CSS rules
server/tests/routes_math_round_trip.rs (new)	Route-level edit/quote round-trip
coverage

Branch is pushed and both commits are in:

• 081543c feat(messages): server-side LaTeX -> MathML rendering module (LC-59) (module +
  dep + CSS)
• 469dbd8 feat(messages): wire LaTeX math into markdown pipeline + round-trip tests
  (LC-59) (wiring + tests, with the merge call-out)

## Summary Server-rendered LaTeX math in message bodies. Inline `$x^2$` and display `$$\int_0^1 f(x)\,dx$$` typeset to MathML via [`pulldown-latex`](https://crates.io/crates/pulldown-latex) (pure Rust, no JS engine, no client-side dependency, no separate stylesheet). MathML is natively accessible to screen readers and renders via the browser's built-in math layout, fitting the existing server-renders-HTMX-swaps architecture. Detection runs inside `markdown::render_inner`'s `Event::Text` branch, so code spans and code blocks remain literal by the same mechanism that already protects mentions and emoji. ## Decision log (load-bearing surfaces) **Dependency choice: pulldown-latex pinned to `=0.7.1`.** Only candidate that satisfies every hard constraint (no client JS, no runtime JS engine, no separate stylesheet, SR-accessible via MathML, pure Rust). `katex-rs` (xu-cheng) requires a JS engine at runtime and HTML+CSS output, ruling it out on two of the constraints. `mathyank` doesn't exist on crates.io; the closest analogue (`RaTeX`) outputs Canvas/PNG/SVG (image-based, ticket explicitly rejected). The pin is exact (`=`, not `^`) so a routine `cargo update` cannot upgrade. Re-run the LC-59 spike before merging any version bump. **Macro recursion is a process-abort vector. The blocklist is the primary mitigation.** Spike-confirmed: `\def\x{\x\x}\x` stack-overflows the parser, and Rust's runtime stack-overflow handler calls `abort()` unconditionally - uncatchable by `catch_unwind`, and a bounded-stack worker thread does NOT contain it either. `views::math::blocklist_re` rejects any span containing `\def`, `\edef`, `\xdef`, `\gdef`, `\let`, `\futurelet`, `\newcommand`, `\renewcommand`, `\providecommand`, `\href`, `\url`, `\csname`. Word-boundary regex, whitespace-tolerant. `\let` is load-bearing: aliasing closes the chain. Denylist, not a proof - strong but version-pinned. **Deep nesting is safe at any reasonable depth.** Parser is iterative, not call-stack recursive - spike-confirmed to depth 6400. **Four failure modes converge on literal-fallback**: caps exceeded, blocklist match, `push_mathml` returns `Err`, output contains `<merror>`. Plus a fifth: `catch_unwind` around the render call catches non-stack-overflow panics. Per-render `Storage` + `Parser` = no shared state. **Display-mode CSS.** `math[display="block"] { margin: 0.5rem 0; }`. We do NOT set `display: block` on display math: that CSS property overrides the browser's native MathML layout and causes children to stack vertically (eyeball-discovered). The `display="block"` ATTRIBUTE already makes the element a centered block via native rendering. ## Unplanned shared-behavior change: the Text-event merge `pulldown_cmark` splits a single text run into multiple `Event::Text` events at backslash-escape boundaries. Math detection per-event therefore could not find any span containing a LaTeX escape. The fix merges consecutive `Event::Text` events into one before the inline-rewrite filter_map runs. Safe across code-block boundaries by construction. The merge ALSO changes how mention / emoji / URL detection sees text at backslash boundaries: a single contiguous string rather than two halves. Mentions: benign. URLs: common shape works identically. See known-limitations for one edge case. ## Known limitations 1. **URL abutting a backslash-escape goes from one anchor to zero.** Source `https://example.com\!end` becomes `https://example.com!end` after cmark, which `linkify` rejects. Pre-merge would have found the URL in the first half-event. Pinned by `bare_url_abutting_backslash_escape_no_anchor_documented_limit`. Not chat-realistic; text is preserved (not lost, not corrupted, not a security hole). 2. **Math inside markdown link labels is intentionally not typeset in v1.** 3. **Math spanning inline-markdown boundaries (`$x_*y*$`) is not detected.** Single-text-event scoped. 4. **Backslash-escaped `$` in chat text (`\$`) is not a literal-dollar escape.** Users wanting a literal dollar should use a code span. 5. **User-defined macros are blocked.** Not supported in v1. ## Test plan - [ ] **WebView2 hand-test (Windows)**. Build the desktop binary on Windows, run, paste the sample below into a room. Pass criterion: every typeset case visually correct. Specifically verify the corrected display-CSS: the display integral renders horizontal and centered, NOT as a vertical character-by-character stack. - [ ] **WKWebView hand-test (macOS)**. Same procedure if a Mac is available. - [x] **WebKit2GTK** was confirmed pre-merge via the LC-59 spike. Sample message for the hand-test: ``` Inline: $x^2 + y^2 = z^2$ Sum: $\sum_{i=1}^{n} i = \frac{n(n+1)}{2}$ Integral: $\int_0^\infty e^{-x^2}\,dx = \frac{\sqrt{\pi}}{2}$ Greek: $\alpha, \beta, \gamma, \Sigma, \Omega$ Sub/sup stack: $a_i^j$ Fraction: $\frac{a + b}{c - d}$ Display: $$\int_0^1 f(x)\,dx$$ Mixed: pre $x^2$ between $\sqrt{y}$ post With mention: $E = mc^2$ thanks @someone Inside inline code: `$x^2$ literal` ``` Automated coverage in this PR: 41 math-module unit tests, 9 markdown integration tests, 4 route-level round-trip tests. ## Files | File | Notes | |---|---| | `server/Cargo.toml` | `pulldown-latex = "=0.7.1"` exact pin | | `server/src/views/math.rs` (new) | Scanner, render boundary, blocklist, caps, `catch_unwind`, `<merror>` detection, 41 tests | | `server/src/views/mod.rs` | `pub mod math;` | | `server/src/views/markdown.rs` | Text-event merge, Event::Text wiring, 9 new tests | | `server/assets/main.css` | Two math CSS rules | | `server/tests/routes_math_round_trip.rs` (new) | Route-level edit/quote round-trip coverage | Branch is pushed and both commits are in: - 081543c feat(messages): server-side LaTeX -> MathML rendering module (LC-59) (module + dep + CSS) - 469dbd8 feat(messages): wire LaTeX math into markdown pipeline + round-trip tests (LC-59) (wiring + tests, with the merge call-out)

longjacksonle added 2 commits 2026-05-24 19:13:25 +02:00

feat(messages): server-side LaTeX -> MathML rendering module (LC-59) ... 081543c553

Adds `views::math` for detecting inline `$...$` and display `$$...$$` math spans in message bodies and rendering them to MathML via `pulldown-latex`. This commit only introduces the module, the pinned dependency, and the CSS rule; the markdown-pipeline wiring lands in the follow-up commit so the chokepoint surface can be reviewed in isolation.

Module shape:
- `render_math_in_text` scans text into Text/Math chunks via pandoc-style boundary rules (open follows non-whitespace, close precedes non-whitespace, close not followed by digit, content non-empty and `$`-free). Non-math chunks route through the existing `render_body` pipeline unchanged; math chunks render to MathML.
- Three caps bound per-message work: `MATH_MAX_SPAN_CHARS = 1024`, `MATH_MAX_SPANS_PER_MESSAGE = 32`, `MATH_MAX_TOTAL_CHARS = 4096`. They count attempts, not successes, so a flood of blocklist-hit or malformed spans cannot consume arbitrary work.
- `try_render_math` is the single render boundary: caps + blocklist + per-render `Storage` + `push_mathml` + `<merror>` substring scan + `catch_unwind`. All five failure modes converge on `None`; the caller substitutes literal escaped text including the original `$` delimiters.

Safety story (the load-bearing surface):
- `pulldown-latex` 0.7.1 implements `\def`-family macros whose unbounded expansion stack-overflows the process. Stack overflow is uncatchable by `catch_unwind` in Rust (the runtime's handler is unconditional abort()), and bounded-stack worker threads do not help either - confirmed empirically in the LC-59 spike. Macro recursion is therefore prevented PRE-RENDER by a control-sequence blocklist: `\def`, `\edef`, `\xdef`, `\gdef`, `\let`, `\futurelet`, `\newcommand`, `\renewcommand`, `\providecommand`, plus `\href`, `\url`, `\csname` as future-proofing. Word-boundary regex so `\definecolor` is not blocked, whitespace-tolerant so `\def \x` is blocked (spike-confirmed evasion).
- `\let` is load-bearing in the blocklist, not optional: the spike confirmed `\let\foo=\def \foo\x{a}\x` is a real alias-evasion against a `\def`-only blocklist.
- It is a denylist, validated against pulldown-latex 0.7.1's specific macro surface and the evasions tested in the spike. Strong, not a proof. The dependency is pinned to `=0.7.1` (exact pin, not caret) so a routine `cargo update` cannot silently invalidate it; re-run the spike before merging any version bump.
- Deep nesting (`\frac`, `\sqrt`, braces) is safe at any depth the length cap permits: the parser is iterative, not call-stack recursive on input structure - spike-confirmed to depth 6400 with no overflow.

CSS rule (assets/main.css):
- `math[display="block"] { margin: 0.5rem 0; }` and `math { vertical-align: -0.15em; }`. We do NOT set `display: block` on display math: that CSS property overrides the browser's native MathML layout and causes children to stack vertically. The `display="block"` ATTRIBUTE (which pulldown-latex sets) already makes the element a centered block via native rendering. Eyeball-confirmed in WebKit2GTK; the WebView2 / WKWebView confirmation is the pre-merge hand-test gate (Chromium's MathML Core handles display-mode layout slightly differently from WebKit).

Tests: 41 unit tests under `views::math::tests` covering scan (basics + adjacent-delimiter battery + pandoc boundary rules + best-effort recovery on malformed display), the render boundary (success, oversize, merror), the blocklist (every entry hits, `\definecolor` does NOT match, whitespace evasion blocked, `\let` aliasing blocked), and integration smoke (pure-text fast-path, math+mention coexistence, `$@user$` mutual-exclusion, cap enforcement, literal-fallback).

Wiring into `markdown::render_inner` and the route-level edit/quote round-trip tests land in the next commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

feat(messages): wire LaTeX math into markdown pipeline + round-trip tests (LC-59) ... 469dbd8f5e

Wires `views::math::render_math_in_text` into `markdown::render_inner`'s `Event::Text` branch (the non-link-label path), replacing the previous direct `render_body` call. The math pass owns chunks inside `$...$` and `$$...$$`; everything else flows through `render_body` unchanged, so mentions / emoji / URL linkification keep working identically for non-math content. Math inside inline code (`Event::Code`) and fenced code blocks (`in_code_block` accumulator) is automatically literal: those events never reach the Event::Text arm where math detection runs - same protection mentions and emoji already have, by construction.

Unplanned shared-behavior change (the merge): pulldown_cmark splits a single text run into multiple `Event::Text` events at backslash-escape boundaries (e.g. `$$\int_0^1 f(x)\,dx$$` emits two events because `\,` ends one and starts the next). Math detection per-event therefore could not find any span containing a LaTeX escape. The fix is to merge consecutive `Event::Text` events into one before the filter_map runs. The merge is safe across code-block boundaries by construction (Start/End(CodeBlock) are non-Text events that flush the accumulator), but it DOES change behavior for non-math inline detection at backslash boundaries:
- Mentions: benign in practice. The mention regex anchors on `^` or whitespace, both preserved across the merge.
- URLs: common shape (URL with whitespace before the escape) works identically. Edge case where a URL abuts a backslash-escape with no separator (`https://example.com\!end` -> post-cmark `https://example.com!end`) goes from one anchor to zero, because linkify rejects the joined string. Not chat-realistic input. Pinned by `bare_url_abutting_backslash_escape_no_anchor_documented_limit`; documented in the merge doc-comment and in the PR description's known-limitations section. The alternative (a smarter merge that preserves URL boundaries at backslash escapes) was rejected as a worse trade: special-case heuristic logic in the hot shared rendering path to handle a shape no one types.

Eight new unit tests in `markdown::tests`:
- Inline / display math typesets through the full pipeline (`math_inline_typesets_in_paragraph`, `math_display_typesets_as_block`).
- Code-protection holds: inline code and fenced code both keep `$x^2$` literal (`math_inside_inline_code_is_literal`, `math_inside_fenced_code_block_is_literal`).
- Math + mention coexist (`math_and_mention_coexist_in_one_paragraph`).
- Math inside link labels is intentionally NOT typeset in v1 (`math_inside_markdown_link_label_is_not_typeset`).
- Backslash-escape merge regression guard (`math_with_internal_backslash_escape_still_typesets`).
- Merge-shared-behavior pinning: mention at backslash boundary works, URL+space at backslash boundary works, URL+abutting-escape is the documented zero-anchor edge case.

Four new route-level integration tests in `routes_math_round_trip.rs` pin the acceptance criterion that edit and quote preserve raw `$...$`, not rendered MathML:
- Edit form for an inline-math message contains the raw source in the textarea (no `<math>`).
- Edit form for a display-math message contains the raw `$$\...\$$` source (no `<math>`).
- Composer-quote chip for a math message contains the raw `$x^2$` source in the excerpt (no `<math>`).
- Round-trip: post -> edit-form -> PATCH same body -> edit-form again. Source survives each cycle.

These are the real coverage of the acceptance criterion. An earlier unit-level `math_source_preserved_for_edit_round_trip` was removed because `render` taking `&str` makes the round-trip assertion tautological at the unit layer.

Pre-merge hand-test gate (NOT yet exercised, see PR description for details): visual confirmation that the chat-math subset renders correctly under WebView2 (Windows desktop wrapper) and WKWebView (macOS), including the corrected display-mode CSS rule. WebKit2GTK (Linux dev-desktop) was confirmed during the spike; the spike artefact has been deleted now that the rule is verified.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

longjacksonle merged commit 6eaaa1ae56 into main 2026-05-24 19:14:28 +02:00

longjacksonle deleted branch feat/lc-59-math 2026-05-24 19:14:28 +02:00

longjacksonle referenced this pull request from a commit 2026-05-24 19:14:28 +02:00

Merge pull request 'feat/lc-59-math' (#193) from feat/lc-59-math into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

psa-systems/lets-chat!193

No description provided.