Every figure,cited.

The mistake. Adviser records, positions, and AUM figures loaded into a vector store, then the assistant is asked to find advisers over $1 billion in New York.

What happens. A vector index answers “what is this about.” It cannot filter, sum, join, or match exactly. It returns advisers near the words “large” and “New York” and misses half the real answers.

What we do. Anything you would put in a WHERE clause or a SUM lives in a relational store. Anything described as “about” or “similar to” lives in a vector store. Most real questions need both: filter in SQL, rank by vector, join on the identifier.

The mistake. The default model from the AI vendor, chosen the way one picks a database driver.

What happens. A model trained on web text places “we reduced our delta exposure” near airline content. It truncates silently at its token limit, so the qualifying footnote is never embedded. When the vendor retires the model, the whole corpus must be re-embedded, because vectors from different models cannot be mixed.

What we do. Start from a strong general model and adapt it on financial text and on question-passage pairs your analysts wrote. Prefer a model you can run yourself, so the retirement date is your decision. Record the model version on every vector.

Read the field note

The mistake. Every document cut into 500-token pieces with 50 tokens of overlap, because a tutorial said so.

What happens. Too small, and a side letter's MFN clause is separated from the carve-out in the next paragraph that reverses it. Too large, and the answering sentence is five percent of the chunk, so the embedding describes the general subject and the match is weak. Either way the passage is about the right topic and says the wrong thing.

What we do. Segment by structure: clause, section, table, footnote. A small segment for matching, the parent returned for reading, so the qualifier travels with the claim. Tables are never flattened into prose.

The mistake. A backtest or a diligence memo reads a manager's AUM history from a table that stores only the current figure for each period.

What happens. Firms restate. A figure filed in March 2025 replaces the one filed in March 2024 for the same period. Your model sees the corrected number on a date when nobody could have known it. The backtest looks better than any real strategy could have been.

What we do. Every row carries a known-on date and a superseded-on date. “What was true on 2024-06-30” is a query, not an assumption. De-registered firms stay in the history.

Read the field note

The mistake. A vector-only index asked to retrieve documents about 037833100.

What happens. Identifiers are strings with no meaning. The model has no idea this one refers to a specific company. It returns passages with similar-looking numbers about other companies.

What we do. Every index is hybrid. A keyword index for exact terms and identifiers, a vector index for meaning, and a fusion step that combines the two rankings.

Read the field note

The mistake. The system returned plausible passages in a demo, and nobody has measured it since.

What happens. Every later change, a new model, a new chunk size, a new document type, is judged by whether the demo still looks good. The system drifts and nobody knows which way.

What we do. Two hundred question-passage pairs written by the analysts who will use the system, before any model is chosen. Recall at five and at ten, run before and after every change.

The mistake. One index for the whole firm, because the assistant “only retrieves what is relevant.”

What happens. Vector stores have no row-level security. A junior analyst asks about compensation and retrieves the passage from the CIO's memo. The assistant quotes it, with a citation.

What we do. Every segment carries the access control of its source document as metadata. Permissions are applied as a filter before ranking, using the identity of the person asking.

The mistake. Three months fine-tuning a model on fund documents so it will “know” them.

What happens. Fine-tuning changes how a model writes far more reliably than what it knows. The tuned model gives fluent, house-style answers about the documents, some of them invented, with no citation, because there is no retrieved passage to cite.

What we do. Retrieval for facts, fine-tuning for form. A memo in your template, a classification into your taxonomy, an extraction from a weekly document: tuning earns its cost. “Answer questions about our documents” does not.

The mistake. The corpus was embedded at launch. Documents have been amended since. The embeddings have not.

What happens. The assistant retrieves a segment from the version of the LPA that was superseded eight months ago, and answers from it with confidence.

What we do. Every segment stores a hash of its source. When the source changes, the affected segments are re-embedded and the old vectors retired. Every vector records the model that produced it.