When the right answer is buried in the pile

Imagine an adjuster working a complex bodily injury claim — hundreds of pages across a demand packet, medical records, a police report, and prior correspondence. The adjuster asks the AI assistant: "What's the incident summary?"

The system retrieves five document chunks, passes them to the language model, and produces a confident, well-structured response — pulled from a preliminary accident report that was later superseded by an amended police narrative documenting a different sequence of events.

No error. No flag. Just a fluent answer built from the wrong source.

This is not a rare edge case in Bodily Injury (BI) claims processing. It is a predictable consequence of how most Retrieval Augmented Generation (RAG) systems retrieve information — and it has real downstream consequences: misstated reserves, incorrect liability assessments, and decisions made on facts that have since been corrected.

Initial intake note Fee schedule Medical records Demand packet Preliminary report Amended police narrative Correct answer — buried AI assistant query "What's the incident summary?" Wrong result Preliminary report

The same problem appears across different query types. Ask about causation instead of the incident, and the system faces an equally treacherous choice — this time between a treating physician's causal opinion and a billing record that attributes the condition to a pre-existing cause. Both contain the right clinical vocabulary. Only one determines coverage. The adjuster asks the AI assistant: "What caused the claimant's lumbar injury?" and gets an irrelevant response from a billing record.

Vector similarity and genuine relevance are not the same thing — and in complex claims, that gap gets expensive.

The good news is there is a targeted fix that does not require rebuilding your AI stack from scratch. It is called a custom reranker, and it is one of the most effective investments you can make in the accuracy of a claims AI assistant.

How standard retrieval works — and where it breaks down

Before understanding the fix, it helps to understand the problem precisely.

Standard RAG retrieval works by converting both a user's query and every document chunk in your knowledge base into dense numerical vectors — mathematical representations of meaning — and returning the chunks that are geometrically closest to the query. The intuition is reasonable: text that means similar things should cluster together, so relevant content should float to the top.

In practice, this works well for general-purpose questions. It struggles in the specialized, document-heavy world of BI claims. A few failure modes show up consistently:

Semantic drift. A query about "causation" in a soft-tissue injury context might retrieve medical record chunks that mention causation in the context of billing codes — semantically related, operationally irrelevant. The embedding model does not distinguish between "causation" as a legal standard and "causation" as a clinical annotation.

Document type conflation. A BI claims knowledge base contains very different kinds of documents: coverage guidelines, treatment records, demand letters, IME reports, fee schedules, and legal correspondence. Standard retrieval treats all of these as equal candidates for any query. An adjuster asking about liability exposure should not be getting back chunks from a medical billing summary — but without document-type awareness, they might.

Chunking artifacts. Medical records and police reports are long, narrative documents where the most important information — a diagnosis, a fault determination — often appears deep in the document after pages of preamble. Fixed-length chunking strategies can split this critical content across boundaries, leaving each half scoring below the relevance threshold individually.

To understand why, it helps to see how the similarity score is actually computed. A common similarity metric, Cosine similarity measures the angle between two embedding vectors — a score of 1.0 means identical direction, 0 means completely unrelated. Scores above 0.80 are typically treated as "highly relevant" and surfaced to the language model. The formula is straightforward:

Here is what that looks like with a real BI query — and why a small score difference hides a large practical difference:

The underlying issue is that cosine similarity measures how similar two pieces of text are, not how useful one is to the other given a specific task. For a claims assistant making adjudication-relevant judgments, that difference matters enormously.

The reranker: a second opinion that changes everything

The retrieve-then-rerank pattern separates two jobs that standard RAG conflates: casting a wide net and picking the right fish.

In this approach, the vector search still does its job — quickly retrieving a broad candidate set of, say, the top 20 or 50 most similar chunks. Then a second model, the reranker, looks at each candidate alongside the original query and assigns it a relevance score based on how well it actually answers this specific question. The top-ranked results from the reranker are what gets passed to the language model.

The key difference is that the reranker evaluates the query and document together, which allows it to pick up on subtle but critical signals that independent embedding cannot: Does this chunk answer the question being asked, or does it just share vocabulary with it? Is this the operative version of the information, or an earlier draft? Is this the right document type for this kind of question?

An off-the-shelf reranker already improves accuracy meaningfully over raw vector search. But a custom reranker — fine-tuned on your own claims data and adjuster feedback — goes significantly further. It learns the relevance patterns specific to your document types and your adjusters' actual queries.

In a BI claims context, that means the reranker can learn distinctions like:

• An amended police report outranks an initial incident log for questions about fault

• A treating physician's narrative outranks a billing summary for questions about injury severity

• A demand letter's liability argument outranks a coverage guideline for questions about exposure

These distinctions are invisible to a general-purpose model. They are learnable by one trained on your data.

The latency cost of adding a reranker is real but modest — typically tens to a few hundred milliseconds over a candidate set. In an environment where an adjuster is already spending hours reviewing a complex BI file, that overhead is negligible against the accuracy gain.

This is precisely the kind of domain intelligence that ClaimLens™ is built to encode. The ClaimLens™ platform combines a curated, insurance-specific dataset with a proprietary Risk Signal Hub to ensure that retrieval is calibrated to the actual decision-making needs of claims and legal professionals — not the average behavior of a general-purpose web corpus.

Even the best reranker is not infallible

A custom reranker dramatically improves retrieval precision — but it does not eliminate error. No retrieval system does.

In complex BI claims, the stakes of a retrieval mistake are high. A reranker trained on general adjuster feedback may not have seen enough examples of a rare claim type — a pedestrian knockdown with disputed crossing signals, or a multi-vehicle pile-up with overlapping coverage layers — to rank correctly every time. Document quality varies: a handwritten physician note, a poorly scanned police supplement, or an unusually structured demand letter may not chunk and embed cleanly regardless of how well the reranker is calibrated. And even a correctly ranked chunk can be misread by the language model if the surrounding context is ambiguous.

This is why retrieval accuracy, however good, is a necessary but not sufficient condition for a trustworthy claims AI. The other half of the equation is transparency: every finding the system surfaces needs to be traceable back to a specific passage in a specific document, so the adjuster can verify it directly.

Think of it the way a good attorney thinks about evidence. The argument may be compelling, but it only holds up if you can point to the exhibit. An AI assistant that tells an adjuster "the treating physician documented a herniated disc at L4-L5" without citing the exact sentence in the medical record is asking for trust without providing grounds for it. When that claim turns out to be wrong — pulled from an early triage note rather than the definitive MRI report — there is no audit trail, no way to catch the error before it propagates into a reserve decision.

Citation-backed outputs change that dynamic entirely. When every statement in a generated summary links directly to the underlying source passage, the adjuster is no longer asked to trust the AI blindly — they are given the tools to validate it in seconds. The AI becomes a navigator, not an oracle: it gets you to the right page of the right document faster than manual review ever could, but the final judgment stays with the professional who can read the exhibit in context.

This is a core design principle in ClaimLens™. Every insight, summary, and recommendation the platform generates is accompanied by citations pointing back to the exact document and passage from which it was extracted. A 90%+ comprehension accuracy rate matters precisely because the remaining cases can be caught — by adjusters who have the source material right in front of them, not buried three folders deep in a claims management system.

Retrieval precision gets you to the right answer most of the time. Cited evidence lets you verify it every time.

Accuracy you can stake decisions on

A custom reranker shifts a BI claims AI assistant along the dimension that matters most in adjudication: not whether the system can find something relevant, but whether it consistently finds the right thing.

The difference shows up in the outputs adjusters depend on. A demand packet summary built from correctly ranked source material captures the operative medical findings, not the billing preamble. A liability assessment grounded in the amended police narrative, not the initial intake note, reflects the actual facts of record. A coverage analysis that surfaces the right policy provision, not a tangentially related guideline, supports a defensible decision.

What you gain is more than accuracy. You gain the kind of trust that lets claims organizations expand AI use into higher-complexity, higher-stakes files. When adjusters learn that the system reliably surfaces the right document at the right moment, they bring harder questions to it. That is the real unlock.

A prototype retrieves what's similar. A production-grade claims assistant retrieves what's useful — and learns, over time, to know the difference.