Vector Databases Compared: pgvector, Pinecone, Weaviate, Qdrant
Which vector database should you use? The answer depends on your scale, stack, and query requirements. Here is the tradeoff breakdown.
The Decision Framework First
Before comparing products, establish requirements:
| Question | Why It Matters |
|---|---|
| How many vectors? | Changes index type and hosting approach |
| What query latency is required? | Determines whether managed cloud or self-hosted is viable |
| What is your existing infrastructure? | pgvector adds zero new services for Postgres teams |
| Do you need filtered search? | Not all databases handle filter + vector search equally well |
| What is the team's operational capacity? | Self-hosted databases require maintenance |
pgvector: Best Default for Most Teams
pgvector is a PostgreSQL extension that adds vector storage and ANN search to a standard Postgres database.
Best for: Teams already using PostgreSQL, datasets up to ~10M vectors with appropriate indexing, applications that need vector search alongside relational queries (filtering by metadata with SQL WHERE clauses).
sql-- Install extension CREATE EXTENSION IF NOT EXISTS vector; -- Create table CREATE TABLE embeddings ( id BIGSERIAL PRIMARY KEY, content TEXT, metadata JSONB, embedding vector(1536) ); -- Create HNSW index for fast ANN search CREATE INDEX ON embeddings USING hnsw (embedding vector_cosine_ops) WITH (m = 16, ef_construction = 64); -- Query SELECT id, content, 1 - (embedding <=> $1::vector) AS similarity FROM embeddings WHERE metadata->>'category' = 'support' -- SQL filter works natively ORDER BY embedding <=> $1::vector LIMIT 10;
Limitations: Performance degrades above 10M vectors without careful tuning. Not purpose-built for vector workloads - a purpose-built vector database will outperform it at scale.
Pinecone: Managed, Zero-Ops
Pinecone is a fully managed vector database. You call an API; they handle infrastructure, scaling, and indexing.
pythonfrom pinecone import Pinecone, ServerlessSpec pc = Pinecone(api_key="your-api-key") # Create index pc.create_index( name="rag-index", dimension=1536, metric="cosine", spec=ServerlessSpec(cloud="aws", region="us-east-1") ) index = pc.Index("rag-index") # Upsert index.upsert(vectors=[ {"id": "chunk_1", "values": embedding_list, "metadata": {"source": "doc1.pdf", "page": 3}}, ]) # Query results = index.query(vector=query_embedding, top_k=10, filter={"source": {"$eq": "doc1.pdf"}})
Best for: Teams that want zero infrastructure management, rapid prototyping, and are willing to pay for the convenience. Pricing is per vector per month.
Limitations: Vendor lock-in. Cost scales with vector count. Query latency is network-dependent (you are calling a remote API). Metadata filtering is less flexible than SQL.
Weaviate: Hybrid Search Built In
Weaviate natively supports hybrid search - combining vector search (dense) with keyword search (BM25, sparse) in a single query.
pythonimport weaviate from weaviate.classes.query import HybridFusion client = weaviate.connect_to_local() # Hybrid query collection = client.collections.get("Documents") response = collection.query.hybrid( query="machine learning production", alpha=0.7, # 0 = pure BM25, 1 = pure vector fusion_type=HybridFusion.RELATIVE_SCORE, limit=10 )
Best for: Applications where keyword + semantic hybrid search outperforms either alone (often true for technical documentation and product search). Has a GraphQL query interface that enables complex filtered queries.
Limitations: More complex to set up than pgvector or Pinecone. GraphQL interface has a learning curve.
Qdrant: Performance and Filtering Precision
Qdrant is a purpose-built vector database that performs well on filtered vector search - queries where you need to filter by metadata and then search within the filtered subset.
pythonfrom qdrant_client import QdrantClient from qdrant_client.models import Filter, FieldCondition, MatchValue client = QdrantClient(url="http://localhost:6333") # Search with precise filtering results = client.search( collection_name="documents", query_vector=query_embedding, query_filter=Filter( must=[ FieldCondition(key="category", match=MatchValue(value="legal")), FieldCondition(key="year", range={"gte": 2020}), ] ), limit=10, )
Best for: High-QPS production workloads, filtered search performance, self-hosted deployments with strong operational capabilities.
Limitations: Requires self-hosting (managed cloud is available but less mature than Pinecone). More operational overhead than pgvector for teams without dedicated infrastructure.
The Recommendation
| Situation | Recommendation |
|---|---|
| New project, team uses Postgres | pgvector |
| Rapid prototype, no infra team | Pinecone |
| Hybrid (keyword + semantic) search needed | Weaviate |
| High QPS, filtered search, self-hosted | Qdrant |
| > 100M vectors, large enterprise | Dedicated vector DB + data engineering team |
Start with pgvector. It handles most RAG use cases without adding a new service. Migrate to a dedicated vector database when you have a concrete performance requirement that pgvector cannot meet.
Common Mistakes
Choosing a dedicated vector database before trying pgvector in Postgres. pgvector handles tens of millions of vectors with acceptable recall and query latency for most production workloads, with zero additional operational overhead. Adding a dedicated vector database is an operational cost (another cluster, another backup strategy, another failure mode). Only graduate to a dedicated system when you have profiled pgvector and found a concrete limitation.
Benchmarking only on small datasets where all options perform similarly. At 10K vectors, FAISS, pgvector, Pinecone, and Weaviate are indistinguishable in latency and recall. The differences that matter emerge at 10M+ vectors under production QPS. Always benchmark at the data scale you expect to operate at, not at the scale that is convenient to test locally.
Ignoring operational overhead in the "best ANN accuracy" comparison. A vector index with 99.5% recall is not better than one with 98% recall if the former requires 3x more memory, manual index rebuilds, and a dedicated ops rotation. Factor in indexing time, memory footprint, managed versus self-hosted trade-offs, and your team's operational capacity when comparing options.
What to Practice Next
- Set up pgvector in Postgres and index 1M synthetic vectors; measure recall@10 and query latency at 100 QPS, then compare against FAISS IndexFlatL2 on the same dataset.
- Read the ANN-benchmarks leaderboard (ann-benchmarks.com) and identify which index types dominate the recall vs. queries-per-second Pareto frontier for your approximate data dimensionality.
- Write a one-paragraph decision rule for when you would upgrade from pgvector to a dedicated vector database, citing the specific metrics and thresholds that would trigger the switch.
Stay in the loop
Get new ML/AI lessons in your inbox.
No account needed. We will send curriculum updates, launch notes, and practical learning resources.
Related Posts
More postsFine-Tuning and Post-Training: LoRA, SFT, DPO, and Reasoning RL
What actually happens after pretraining, and when you should do any of it yourself. Parameter-efficient fine-tuning with LoRA, supervised fine-tuning data, preference optimization, and the reinforcement learning recipe behind reasoning models, with a decision framework and a project you can run on one GPU.
LLM Context Windows: What They Mean for System Design
Context window size shapes every architectural decision in LLM applications. This post covers how to reason about context allocation, the limits that still matter even with large windows, and the patterns that scale.
Common ML Architectures Reference: CNN, RNN, Transformer, MoE
A concise technical reference for the neural network architectures that power modern ML - what each one does, how it works, when to use it, and what to watch out for.