How semantic search works in 2026 — embeddings, vectors, and hybrid retrieval explained plainly. Why keyword search keeps missing your saved bookmarks, and what fixes it.
Type "how much does it cost to acquire customers" into a saved-bookmarks search and watch nothing match. The video you're looking for is right there in your library — you saved it. The speaker said "CAC" sixteen times. Your search said "customer acquisition cost." The words don't overlap. The match fails.
This failure mode has a name: the vocabulary mismatch problem. It's why keyword search keeps losing, and it's why semantic search is now the default in every serious tool that searches anything text-shaped. Below: how semantic search actually works in 2026 — plain English, no math degree required — and why most consumer apps that ship "AI search" still get it wrong.
Old search systems matched literal strings. You typed customer acquisition cost, the system found documents containing those exact words. Documents that said "CAC" or "the cost of bringing in new buyers" or "what we pay per signup" stayed invisible.
In information retrieval research this has been measured for forty years. Typical vocabulary mismatch is somewhere around 70-80%: people use different words for the same concept most of the time. If your search only matches literal strings, you miss four out of five relevant documents. That's a polite way of saying "keyword search has been broken since 1985."
Two things fixed it. The first is embeddings: a way to represent meaning as numbers, so two phrases that mean the same thing end up "close" even if they share no words. The second is vector databases: storage and indexes that can find the closest vectors fast, even when the corpus is millions of documents.
Together, they're what makes "semantic search" a thing.
Take a sentence. Run it through a neural network. The network returns a list of numbers — typically 384, 768, or 1,536 of them. That list is the embedding of the sentence.
The trick is that the network is trained so that sentences with similar meanings produce similar lists. The literal example:
"What is customer acquisition cost" → embedding A"How much do we pay to bring in a new customer" → embedding B"How are seagulls today" → embedding CEmbeddings A and B will be close in the 1,536-dimensional space they live in. C will be far away. Closeness is measured by cosine similarity — how aligned the two vectors are in that high-dimensional space. Numbers near 1.0 mean almost identical meaning; numbers near 0 mean unrelated.
You can think of it as plotting every phrase as a point in a 1,536-dimensional galaxy where direction encodes meaning. The actual coordinates are meaningless on their own — what matters is which other phrases land near which.
Several models are competitive in 2026. OpenAI's text-embedding-3-small ships 1,536 dimensions natively and supports Matryoshka representation (you can truncate to 768 or 256 dimensions for storage savings without retraining). Cohere's embed-v3 is competitive on multilingual content. Open-source alternatives like bge-large and nomic-embed-text are within a few accuracy points and run on a Mac.
For multilingual workloads — like saved-videos in 30 languages with queries in any of them — text-embedding-3-small is currently the best balance of accuracy, cost, and dimensions. At SavedThat we use the 768-dimension Matryoshka truncation: half the storage, ~99% of the search quality. (Read more on Matryoshka representation.)
Plenty of consumer apps in 2026 advertise "AI search." Most of them mean one of three things:
Only the first one actually solves vocabulary mismatch on your data. The other two are productised but not useful for finding what you saved.
A quick diagnostic: if you can search for the concept and not find the document, but you can find it by searching for one of its specific words, the tool is doing #2 or #3 wrapped as "AI search." If you can find the document via paraphrase, it's #1.
Embeddings on their own don't help if you have to compare every query against millions of vectors one at a time. The full search of a 50,000-document index takes hundreds of milliseconds the naive way — way too slow for interactive use.
The fix is approximate nearest neighbour (ANN) indexing. Specialised data structures — HNSW (Hierarchical Navigable Small World), IVF, ScaNN — store vectors so the closest matches can be found in under 10 milliseconds even on libraries with tens of millions of vectors.
The current state of the art (May 2026) for production semantic search:
pgvector extension; standalone vector DBs like Qdrant, Pinecone, and Weaviate all use HNSW or close variants. SavedThat runs HNSW on pgvector because keeping vectors next to the relational metadata is operationally simpler than maintaining two databases.halfvec(768) type in pgvector is now the default for any cost-conscious deployment.For SavedThat-scale (currently ~120 saved videos × ~50 chunks each = 6,000 vectors), the index fits in RAM and lookups are sub-millisecond.
Now the punchline that most tutorials skip: semantic search loses to keyword search on exact phrases.
If you search for "text-embedding-3-small" (an exact product name), keyword search returns the document mentioning that exact string at rank 1. Semantic search may rank a document that talks about "OpenAI embeddings in general" higher because it's semantically closer to the average meaning of the query, even though the literal string is missing.
This is real. Public benchmarks consistently show pure semantic underperforms keyword for queries with high specificity: product names, function signatures, exact quotes, dates, brand names. The thing semantic was supposed to fix (vocabulary mismatch) doesn't exist for those queries because the user typed the literal string they wanted.
The fix is to run both and merge the results.
Hybrid retrieval is the technique that's quietly become the default in serious production search systems since 2024:
1 / (rank + 60), summed across the two methods. Documents that appear well in both get a meaningful boost; documents appearing in only one still contribute.The constant 60 is a magic number that's been benchmarked to death and works across most corpora. You can tune it; you usually don't need to.
The result: paraphrased queries still find concept-matches (via the semantic side), and exact-phrase queries still rank the literal match at #1 (via the keyword side). Each method's blindspot is covered by the other.
This is what SavedThat ships as the default search. The search_chunks RPC in our Postgres runs both queries in parallel, computes RRF scores, and returns ranked hits in ~80ms for our current corpus size. (Microsoft on hybrid search)
For high-stakes use cases — legal discovery, medical literature, enterprise search over millions of documents — hybrid RRF is usually the second-to-last layer. The last layer is re-ranking with a cross-encoder model.
A cross-encoder is a smaller, slower model that takes the full query and the full candidate document and computes a tight relevance score by attending across both jointly. You can't run it on a million documents (too slow); you run it on the top 50-200 results from hybrid retrieval to re-order them precisely.
Cohere ships rerank-v3 for this. Open-source alternatives: bge-reranker-large, jina-reranker-v2. For consumer-scale workloads like SavedThat, the marginal gain doesn't justify the 100ms latency hit. For enterprise legal-discovery search, cross-encoder re-ranking is now standard.
We'll add it when our corpus or our use case demands it. Not before.
The whole pipeline is roughly:
| Stage | Cost |
|---|---|
| Embedding 1,000 chunks (~50 videos) | ~$0.02 in OpenAI credits |
| Storing 1M chunks in pgvector | ~$25/mo on a small Postgres instance |
| HNSW index build for 1M chunks | ~30 seconds, one-time per major data change |
| One search query (embed + HNSW lookup + FTS + RRF) | ~$0.0001 + 80ms |
At SavedThat's pricing, one Pro subscription ($9.99/mo) covers thousands of searches and several hundred new saves. The economics are fine for consumer-scale; what doesn't work is trying to fund this with a free-forever-unlimited plan because the OpenAI bill grows linearly with content while revenue caps at zero.
Three reasons:
halfvec and MRL the math is twice that.The tools shipping semantic search from day one — Mem, Reflect, Glasp's premium tier, SavedThat — are mostly post-2022 builds because that's when the embedding cost dropped below the threshold where consumer-scale could be priced sustainably.
When you evaluate a search-inside-X tool in 2026, ask:
text-embedding-3-small works across 100+ languages; older or smaller models don't.For saving videos and finding them later: SavedThat ships all four (architecture overview here). Glasp ships keyword only (free tier) and semantic (paid). Mem ships semantic only without hybrid. Notion's "AI search" is item 2 — LLM on keyword.
That's the lay of the land in May 2026.
Functionally none — in practice the terms are used interchangeably. Strictly, an embedding is the *representation* a model produces, and a vector is just a list of numbers. Every embedding is a vector; not every vector is an embedding (a column of pixel intensities is a vector but not an embedding). Most papers and engineering docs use 'vector' as the storage term and 'embedding' as the conceptual term.
Depends on the embedding model. OpenAI's text-embedding-3-small was trained on 100+ languages and produces vectors that are roughly aligned across them — a Russian query and an English document about the same concept land in similar regions of vector space. Smaller open-source models often degrade significantly on non-English. If multilingual is a requirement, verify the model card explicitly mentions cross-lingual alignment.
Running two searches and concatenating the results just gives you twice as many results. Hybrid search merges them by score (RRF or weighted fusion) so the final ranking reflects both methods' agreement. A document at rank 1 in keyword and rank 1 in semantic ends up at rank 1 overall; a document at rank 1 in keyword but rank 50 in semantic falls in the middle. Without merging, you can't tell which results to trust.
Embeddings encode semantic meaning, not literal strings. A query like 'text-embedding-3-small' has its meaning in 'OpenAI embeddings family member' more than in the literal token sequence. The vector for the query is similar to vectors for documents discussing OpenAI embeddings broadly, which may rank higher than the specific document mentioning the exact product name. Keyword search has zero ambiguity on exact strings — they either appear or don't. Hybrid covers both modes.
Related but not the same. Semantic search is the retrieval step — find documents close to a query in vector space. RAG (Retrieval-Augmented Generation) is a system that uses semantic search to fetch documents, then passes them to a generative LLM that writes an answer using the documents as context. RAG includes semantic search as one component; you can have semantic search without RAG (when you just want to find the documents, not generate text on top).
For inference — searching against a built index — no. CPU is fine for retrieval at consumer scale. For building the index — running the embedding model on your content — a GPU is fast but not required. OpenAI's API does the embedding in their datacentre, returning the vector for you to store; the only thing on your end is the vector database and the query embedding (one short API call per query). Cost is pennies per thousand searches.
Search inside saved videos by what was actually said — across YouTube, Instagram, and TikTok. How transcript search works in 2026, and four tools that do it.
The 7 best video transcript search tools in 2026, ranked by what they actually do well. SavedThat, Glasp, Otter, Fireflies, Reduct, Trint, plus DIY Whisper.
The best AI video bookmark manager in 2026 depends on what you save. Honest comparison of SavedThat, Mymind, Raindrop, and Glasp — pricing, search, platforms.