Definition
Keyword search matches documents containing the exact terms in a query, while semantic search uses vector embeddings to find documents that are conceptually similar, even if they use different words.
In Depth
Traditional keyword search relies on term frequency, inverted indexes, and algorithms like BM25 to rank documents that contain the query terms. Semantic search uses neural networks to convert text into high-dimensional vectors (embeddings) and finds results based on cosine similarity in that vector space. This means a semantic search for 'affordable accommodation' can match documents about 'budget hotels' or 'cheap places to stay.' In RAG applications, combining both approaches yields the best results: semantic search for recall and keyword search for precision. Search APIs like Scavio return keyword-matched results from major platforms, which can be combined with vector database results in a hybrid retrieval strategy.
Example Usage
A RAG pipeline uses semantic search against a vector database of internal documents and keyword search via Scavio for real-time web results. The two result sets are merged and re-ranked before being sent to the LLM as context, combining institutional knowledge with current information.
Platforms
Semantic Search vs Keyword Search is relevant across the following platforms, all accessible through Scavio's unified API:
- YouTube
Related Terms
Retrieval-Augmented Generation (RAG)
Retrieval-Augmented Generation (RAG) is an AI architecture that enhances large language model outputs by first retrievin...
Search Intent Classification
Search intent classification is the process of categorizing a search query by the user's underlying goal: informational ...
Structured Search Results
Structured search results are search engine results that have been parsed and organized into a machine-readable format l...