Building AI-Powered Search

Keyword search is dead. Users expect search that understands intent, handles synonyms, and finds relevant content even when query terms do not match exactly. AI-powered semantic search, built with Chinese embedding models and LLMs, delivers this at a fraction of the cost of traditional enterprise search solutions. This guide shows you how to build it.

Why Semantic Search Matters

Query	Keyword Search	Semantic Search
"best laptop for coding"	Matches "best", "laptop", "coding"	Finds "developer workstation", "programming notebook"
"how to reduce server costs"	Exact matches only	Finds "cloud cost optimization", "infrastructure savings"
"startup funding advice"	Limited by vocabulary	Finds "seed round tips", "venture capital guide"

Architecture Overview

A production AI search system has three layers:

1. Embedding Layer: Convert documents and queries into dense vector representations
2. Retrieval Layer: Find the most similar vectors using approximate nearest neighbor (ANN) search
3. Reranking Layer: Use a cross-encoder or LLM to refine and reorder results

Step 1: Generate Embeddings with Chinese Models

Chinese embedding models like BGE and Qwen-Embedding produce high-quality vectors for semantic search. They are optimized for Chinese-English mixed text and outperform many Western alternatives on multilingual benchmarks.

Document Embedding

import openai
import numpy as np

client = openai.OpenAI(
    base_url="https://tokenease.io/v1",
    api_key="your-tokenease-key"
)

def get_embedding(text, model="qwen-embedding"):
    response = client.embeddings.create(
        model=model,
        input=text
    )
    return np.array(response.data[0].embedding)

# Embed a document
doc_text = """
TokenEase provides unified access to Chinese AI models including DeepSeek,
GLM, Qwen, Kimi, and Doubao through a single OpenAI-compatible API.
"""
embedding = get_embedding(doc_text)
print(f"Embedding shape: {embedding.shape}")  # (1024,) or (768,)

Embedding Model Comparison

Model	Dimensions	Best For	Cost/1M tokens
Qwen-Embedding	1024	General purpose, multilingual	$0.10
BGE-Large	1024	Chinese text, academic	$0.08
GTE-Large	768	English-focused, fast	$0.06
E5-Multilingual	1024	Cross-lingual search	$0.12

Step 2: Store Vectors in a Vector Database

You need a database optimized for similarity search. Options range from managed services to self-hosted solutions.

Option A: Pinecone (Managed)

from pinecone import Pinecone

pc = Pinecone(api_key="your-pinecone-key")
index = pc.Index("my-search-index")

# Upsert documents
index.upsert(vectors=[{
    "id": "doc_001",
    "values": embedding.tolist(),
    "metadata": {"title": "TokenEase Guide", "category": "tutorial"}
}])

Option B: Qdrant (Self-Hosted)

from qdrant_client import QdrantClient

client = QdrantClient(host="localhost", port=6333)

# Create collection
client.create_collection(
    collection_name="documents",
    vectors_config={"size": 1024, "distance": "Cosine"}
)

# Add documents
client.upsert(
    collection_name="documents",
    points=[{
        "id": "doc_001",
        "vector": embedding.tolist(),
        "payload": {"title": "TokenEase Guide", "text": doc_text}
    }]
)

Option C: pgvector (PostgreSQL Extension)

# SQL to set up
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    title TEXT,
    content TEXT,
    embedding vector(1024)
);
CREATE INDEX ON documents USING ivfflat (embedding vector_cosine_ops);

# Query similar documents
SELECT title, content, 1 - (embedding <=> query_embedding) AS similarity
FROM documents
ORDER BY embedding <=> query_embedding
LIMIT 10;

Step 3: Implement Semantic Search

With documents embedded and stored, searching is straightforward:

def semantic_search(query, top_k=10):
    # 1. Embed the query
    query_embedding = get_embedding(query)
    
    # 2. Search vector database
    results = index.query(
        vector=query_embedding.tolist(),
        top_k=top_k,
        include_metadata=True
    )
    
    # 3. Return formatted results
    return [{
        "id": match.id,
        "score": match.score,
        "title": match.metadata["title"],
        "text": match.metadata.get("text", "")[:200]
    } for match in results.matches]

# Example search
results = semantic_search("how to save money on AI APIs")
for r in results:
    print(f"{r['title']} (score: {r['score']:.3f})")
    print(f"  {r['text']}...")
    print()

Step 4: Build Hybrid Search (Keyword + Semantic)

Pure semantic search sometimes misses exact keyword matches that users expect. Hybrid search combines both approaches:

from rank_bm25 import BM25Okapi
import numpy as np

class HybridSearch:
    def __init__(self, documents):
        self.documents = documents
        self.tokenized_docs = [doc["text"].lower().split() for doc in documents]
        self.bm25 = BM25Okapi(self.tokenized_docs)
        
        # Pre-compute embeddings
        self.embeddings = [get_embedding(doc["text"]) for doc in documents]
    
    def search(self, query, top_k=10, alpha=0.5):
        # Semantic scores
        query_embedding = get_embedding(query)
        semantic_scores = [
            np.dot(query_embedding, doc_emb) / 
            (np.linalg.norm(query_embedding) * np.linalg.norm(doc_emb))
            for doc_emb in self.embeddings
        ]
        
        # BM25 scores
        tokenized_query = query.lower().split()
        bm25_scores = self.bm25.get_scores(tokenized_query)
        
        # Normalize scores
        semantic_scores = np.array(semantic_scores)
        bm25_scores = np.array(bm25_scores)
        
        semantic_scores = (semantic_scores - semantic_scores.min()) / \
                         (semantic_scores.max() - semantic_scores.min() + 1e-8)
        bm25_scores = (bm25_scores - bm25_scores.min()) / \
                     (bm25_scores.max() - bm25_scores.min() + 1e-8)
        
        # Combine scores
        combined_scores = alpha * semantic_scores + (1 - alpha) * bm25_scores
        
        # Get top results
        top_indices = np.argsort(combined_scores)[::-1][:top_k]
        
        return [{
            "document": self.documents[i],
            "semantic_score": semantic_scores[i],
            "bm25_score": bm25_scores[i],
            "combined_score": combined_scores[i]
        } for i in top_indices]

# Usage
searcher = HybridSearch(documents)
results = searcher.search("API cost optimization", alpha=0.7)

Step 5: Rerank with a Cross-Encoder

Initial retrieval returns candidates. A cross-encoder reranker scores each query-document pair more accurately, significantly improving result quality.

def rerank_results(query, results, top_n=5):
    pairs = [(query, r["document"]["text"]) for r in results]
    
    # Use LLM as reranker
    rerank_scores = []
    for doc_text in [r["document"]["text"] for r in results]:
        response = client.chat.completions.create(
            model="qwen",
            messages=[{
                "role": "user",
                "content": f"Rate relevance (0-10) of this document to '{query}':\n\n{doc_text[:500]}"
            }],
            max_tokens=10
        )
        try:
            score = float(response.choices[0].message.content)
        except:
            score = 5.0
        rerank_scores.append(score)
    
    # Reorder by rerank score
    for i, r in enumerate(results):
        r["rerank_score"] = rerank_scores[i]
    
    return sorted(results, key=lambda x: x["rerank_score"], reverse=True)[:top_n]

Step 6: Add a Generative Layer (RAG Search)

The final evolution is adding a generative layer that synthesizes answers from retrieved documents:

def rag_search(query, top_k=5):
    # 1. Retrieve relevant documents
    results = semantic_search(query, top_k=top_k)
    context = "\n\n".join([r["text"] for r in results])
    
    # 2. Generate answer with context
    response = client.chat.completions.create(
        model="deepseek",
        messages=[{
            "role": "system",
            "content": "Answer based on the provided context. Cite sources."
        }, {
            "role": "user",
            "content": f"Context:\n{context}\n\nQuestion: {query}"
        }]
    )
    
    return {
        "answer": response.choices[0].message.content,
        "sources": results
    }

# Example
result = rag_search("What is TokenEase and how much does it cost?")
print(result["answer"])
print("\nSources:")
for src in result["sources"]:
    print(f"  - {src['title']}")

Performance Benchmarks

Metric	Keyword Search	Semantic Search	Hybrid Search	Hybrid + Rerank
Recall@10	0.42	0.78	0.85	0.91
NDCG@10	0.38	0.72	0.81	0.88
Latency (ms)	15	45	60	250
Cost/query	$0	$0.001	$0.001	$0.005

Production Best Practices

Chunking Strategy

Break long documents into chunks of 200-500 tokens. Overlap chunks by 50 tokens to preserve context across boundaries.

Metadata Filtering

Use metadata filters to narrow search scope before vector comparison. This reduces latency and improves relevance.

results = index.query(
    vector=query_embedding.tolist(),
    top_k=10,
    filter={"category": {"$eq": "tutorial"}, "language": {"$eq": "en"}}
)

Caching Frequent Queries

Cache embeddings for popular queries. 20% of queries typically account for 80% of search volume.

Monitoring Search Quality

Track these metrics in production:

• Click-through rate (CTR): Are users finding relevant results?
• Mean reciprocal rank (MRR): How high is the first relevant result?
• Zero-result rate: How often do searches return nothing?
• Query latency: Is response time acceptable?

Frequently Asked Questions

How many dimensions do I need?

1024 dimensions is the sweet spot for most applications. Higher dimensions (1536-4096) offer marginal improvements at significantly higher storage and compute costs.

Can I search across languages?

Yes. Qwen-Embedding and BGE-Multilingual handle cross-lingual search natively. A query in English can retrieve Chinese documents and vice versa.

How do I handle document updates?

Delete the old embedding and insert the new one. Most vector databases support atomic upserts. For high-velocity updates, batch them and process every 5-10 minutes.

What about privacy-sensitive data?

Embeddings are one-way transformations — you cannot reconstruct the original text from embeddings. However, queries are sent to the API. For maximum privacy, use self-hosted embedding models.