What Is Vectorization? A Working Explainer for Builders Setting Up RAG

Vectorization is the step that turns your text into numbers a database can search by meaning instead of keywords — and it’s the single most common place a RAG build breaks. Get the model, dimension count, and index type mismatched and you’ll get either a silent 400 error or, worse, retrieval that “works” but returns garbage. This is a working walkthrough, not a definitions page: real models named, real dimension counts, real per-million-token cost, and the two dimension-mismatch errors that eat the most debugging time.

What is vectorization, actually?

Vectorization is running a piece of text through an embedding model that outputs a fixed-length numeric array — the “vector” — positioned in high-dimensional space so that semantically similar text lands close together. “Cheap flights to Seoul” and “affordable Seoul airfare” end up near each other in vector space even though they share almost no keywords. That’s the entire trick behind retrieval-augmented generation (RAG): instead of a SQL LIKE search, you embed the user’s query, find the nearest stored vectors, and hand those chunks to the LLM as context.

Which embedding model should you actually use?

For most builders in 2026, it comes down to three named options: OpenAI’s text-embedding-3 family, Voyage AI’s voyage-3 family, or Cohere’s embed models. Here’s what they cost and ship, per the vendors’ current published specs:

Pro Tip: Start with the small/lite tier from whichever provider you’re already paying for API calls to — switching embedding models later means re-embedding your entire corpus, so the real cost of “just try the big one” is a full re-index, not a config flag.

What does pgvector actually support?

Postgres, via the pgvector extension, is the default choice for teams who don’t want to run a dedicated vector database. The catch that trips up most first-time setups: the standard vector type only supports HNSW indexing up to 2,000 dimensions. If you’re storing OpenAI’s full text-embedding-3-large output at 3,072 dimensions, an HNSW index build on that column will fail outright.

The fix is one of three: truncate to 1,536 dimensions using the model’s dimensions parameter (works for both OpenAI models via Matryoshka Representation Learning), switch to pgvector’s halfvec type (16-bit floats, indexable to 4,000 dimensions, roughly half the storage), or drop the index and accept a slower sequential scan on a small table.

Per pgvector’s own project documentation, the vector type’s HNSW index ceiling is 2,000 dimensions — vectors beyond that require the halfvec type or storing without an index.

Why does a dimension mismatch break your app instead of just being slow?

Because Postgres and most vector databases enforce dimension equality at the column or index level. If your table’s vector column is declared vector(1536) and you insert an embedding from a model that returns 1,024 dimensions, you get a hard insert error — not a bad-but-working result. The nastier version of this bug: you re-embed a subset of documents with a different model version and don’t get an error, because both models happen to output 1,536 dimensions, but the vectors are no longer comparable. Retrieval quality quietly degrades and nothing in your logs tells you why.

Watch out: Never mix vectors from two different models — or the same model at two different dimensions settings — in one searchable column, even if the dimension counts match. Cosine distance between an OpenAI vector and a Voyage vector is meaningless; they don’t share a coordinate space.

How do you actually wire this into a RAG pipeline?

The working version: chunk your source documents (typically 300-800 tokens per chunk with slight overlap), call the embedding API in batches, store each vector alongside its source text and metadata in a Postgres table with a pgvector column, then build an HNSW index on that column once the initial backfill is done — building the index during the bulk insert is what makes a 100K-row backfill take hours instead of minutes. At query time, embed the user’s question with the exact same model and dimension setting, then run a cosine-distance nearest-neighbor query, typically pulling the top 5-8 chunks as context, and hand them to whichever model is drafting the final answer — for long-form synthesis over those retrieved chunks, Claude Sonnet 4.6 has tested well on long-form writing tasks that need to stay grounded in retrieved source text rather than drift into generic filler.

If you’re automating the ingestion side of this pipeline — pulling source documents on a schedule, chunking, embedding, and re-indexing without a human triggering each step — the same orchestration patterns from a lean content automation stack built for one-person teams apply directly: cron the ingestion, but gate the index rebuild behind a manual or scheduled trigger, not every insert.

Pro Tip: Build the HNSW index only after the initial bulk load finishes, not before — indexing during a large insert both slows the insert and produces a worse-quality index than a build-after-load pass. This is documented pgvector guidance, not a guess.

What does this cost at real scale?

A 500-page knowledge base chunked at ~500 tokens per chunk runs roughly 400K tokens through the embedding model — at OpenAI small’s $0.02/1M rate, that’s under a cent to embed the entire corpus once. The real cost driver isn’t the embedding call, it’s re-embedding: every time you change chunking strategy, switch models, or add a new dimension setting, you pay that cost again across the full corpus. Teams that treat embedding as a one-time backfill instead of budgeting for periodic re-indexing are the ones surprised by the bill later. If you’re weighing this against fine-tuning your own retrieval model instead of using an off-the-shelf embedding API, the economics shifted meaningfully after Fireworks AI’s $1.505B raise made managed fine-tuning infrastructure cheaper to access.

Pro Tip: Log token counts per embedding batch from day one. It’s the only way to catch a chunking regression (someone drops the overlap setting, chunk count triples) before the monthly bill does.

Key Takeaways

FAQ

Q: What’s the difference between vectorization and embedding?
A: They’re the same process described two ways — “vectorization” describes the output (a vector), “embedding” describes the technique (embedding meaning into coordinate space). Vendors like OpenAI and Voyage call their APIs “embeddings” endpoints.

Q: Do I need a dedicated vector database like Pinecone or Weaviate?
A: Not to start. pgvector on Postgres handles most RAG workloads up to a few million vectors comfortably, and you avoid running a second database. Dedicated vector databases earn their keep at higher scale or when you need features like multi-tenant namespace isolation out of the box.

Q: Can I use a smaller, cheaper embedding model without hurting retrieval quality?
A: Often yes — Voyage’s voyage-3-lite ships 512-dimension vectors at the same $0.02/1M price as OpenAI’s 1,536-dimension small model, with a smaller storage footprint. The only way to know for your corpus is to run a recall test, not to assume bigger dimensions always win.

Q: What happens if I insert a vector with the wrong dimension count?
A: Postgres rejects the insert outright if the column has a fixed dimension declared (e.g. vector(1536)). That’s the safe failure mode — the dangerous one is two same-dimension-but-different-model vectors landing in the same column with no error at all.

Q: How often should I re-embed my corpus?
A: Only when you change the embedding model, the dimension setting, or your chunking strategy. Re-embedding on a schedule “just in case” burns budget for no retrieval benefit — treat it as a triggered event, not a cron job.

Last updated: 2026-08-12