fix(ai): /ask retrieval OR + stopword drop (follow-up to #636, LC-676) #637

Merged

longjacksonle merged 1 commit from fix/LC-676-ask-retrieval-followup into main

2026-08-04 21:03:01 +02:00

longjacksonle commented

2026-08-04 20:53:55 +02:00

Owner

Follow-up to LC-676 / PR #636.

Why a second PR

PR #636 was merged at its first commit (09426e6), before the retrieval-fix commit (434c0b8) reached the branch - so main got the loading state, prompt, rendering, slash-hint, and composer fixes, but NOT the actual retrieval fix. That retrieval fix is the one that resolves the reported symptom (a room full of "David" still answering "who is david?" with "Could not find information about David"). This PR cherry-picks it onto current main.

The fix

build_context ran the question through sanitize_fts_query, which space-joins terms = FTS5 implicit AND, so "who is david?" required a single message containing "who" AND "is" AND "david" - none exists - and retrieval returned nothing, so the model correctly said "no context".

New db::chat::fts_query_any (used only by the /ask RAG path): trims punctuation, drops common question stopwords, and joins the remaining content words with OR, so "who is david?" -> "david" and matches. The search box keeps its precise AND semantics.

Tests

db_search: a unit test pins the OR + stopword behavior, and an integration test asserts the old AND query retrieves nothing while fts_query_any retrieves exactly the David context. Full db_search (18), fmt green.

🤖 Generated with Claude Code

https://claude.ai/code/session_013pGppokMfgq5W46EY2Takg

Follow-up to LC-676 / PR #636. ## Why a second PR PR #636 was merged at its first commit (`09426e6`), before the retrieval-fix commit (`434c0b8`) reached the branch - so main got the loading state, prompt, rendering, slash-hint, and composer fixes, but NOT the actual retrieval fix. That retrieval fix is the one that resolves the reported symptom (a room full of "David" still answering "who is david?" with "Could not find information about David"). This PR cherry-picks it onto current main. ## The fix `build_context` ran the question through `sanitize_fts_query`, which space-joins terms = FTS5 implicit AND, so "who is david?" required a single message containing "who" AND "is" AND "david" - none exists - and retrieval returned nothing, so the model correctly said "no context". New `db::chat::fts_query_any` (used only by the /ask RAG path): trims punctuation, drops common question stopwords, and joins the remaining content words with OR, so "who is david?" -> `"david"` and matches. The search box keeps its precise AND semantics. ## Tests `db_search`: a unit test pins the OR + stopword behavior, and an integration test asserts the old AND query retrieves nothing while `fts_query_any` retrieves exactly the David context. Full `db_search` (18), fmt green. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_013pGppokMfgq5W46EY2Takg

longjacksonle added 1 commit

2026-08-04 20:53:55 +02:00

fix(ai): /ask retrieval uses OR + stopword drop so it finds room context (LC-676)

check-secrets / Nosey parker (push) Successful in 3s

Details

check-secrets / Kingfisher (push) Successful in 5s

Details

check-secrets / TruffleHog (push) Successful in 6s

Details

check-secrets / Nosey parker (pull_request) Successful in 3s

Details

check-secrets / Kingfisher (pull_request) Successful in 6s

Details

check-secrets / TruffleHog (pull_request) Successful in 6s

Details

Check / clippy + fmt + tests (pull_request) Successful in 9m2s

Details

Create release / Create release from merged PR (pull_request) Has been skipped

Details

87befc86f2

The real reason /ask kept dead-ending even when the room clearly discussed the subject (e.g. a room full of "David" answering "who is david?" with "Could not find information about David"): build_context ran the question through sanitize_fts_query, which space-joins terms = FTS5 implicit AND. So "who is david?" only matched a message containing "who" AND "is" AND "david" - which no real message does - and retrieval returned nothing, so the model correctly reported no context. The prompt phrasing was never the cause here; retrieval was.

Add db::chat::fts_query_any for the RAG path: it trims punctuation, drops common question stopwords (who/is/the/what/...), and joins the remaining content words with OR, so "who is david?" becomes "david" and matches. build_context now uses it; the deliberate search box keeps its AND semantics via sanitize_fts_query. fts_room_context then ranks the candidates as before.

Tests (db_search): a unit test pins the OR + stopword behavior, and an integration test seeds a David message + an unrelated one and asserts the old AND query retrieves nothing while fts_query_any retrieves exactly the David context. Full db_search (18), routes_assistant, fmt, clippy green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013pGppokMfgq5W46EY2Takg

longjacksonle scheduled this pull request to auto merge when all checks succeed

2026-08-04 20:54:31 +02:00