Embedding Models Explained

The support search box answers refund status and returns nothing for how do I get my money back. Nothing is broken: the index matches tokens literally, and no document in the library contains those four words. The answer exists — a policy page titled Account recovery and refunds, written in sentences that share no vocabulary with the question.

An embedding model attacks that failure directly. It maps text to a fixed-length vector of numbers and places texts with similar meaning at nearby coordinates, whatever words they use:

text ──► embedding model ──► [0.021, -0.114, 0.873, ..., 0.045]

  2-D projection of that space

   'how do I get my money back'
            *
            | similar meaning
            *
   'refund policy: credited in 5 days'

   'reset my password'        'SKU-7741-BLK'
          *                         *
    different meaning         exact-string territory

Two properties do the work. The vector is fixed length: a four-word question and a 900-word policy page both come out as 1,536 numbers, so a query and a document can be compared by one arithmetic operation. And the numbers encode meaning rather than spelling — the model learned to connect get my money back with refund during training, not from a synonym table someone maintains by hand.

What comes back is coordinates, not prose: no summary, no keywords, no label. Producing the vector is half the job; turning coordinates into a ranked list needs a similarity metric, and that is where Similarity Metrics for Vectors picks up. First, the mechanics of getting vectors out of a model without corrupting the index.

An embedding is meaningful only relative to the model that produced it. A vector from model A and a vector from model B occupy different coordinate systems — same shape, unrelated meaning, no arithmetic that relates them. The model name is part of the data.

import numpy as np
from openai import OpenAI

client = OpenAI()

docs = [
    'Refunds are credited within 5 business days of approval.',
    'Account recovery: request a fresh sign-in link.',
    'Pricing tiers for annual billing.',
]
query = 'how do I get my money back'

def embed(texts, model='text-embedding-3-small'):
    resp = client.embeddings.create(model=model, input=texts)
    return np.array([d.embedding for d in resp.data])

doc_vectors = embed(docs)
q_vector = embed([query])[0]
print(doc_vectors.shape, q_vector.shape)

Input: the three documents and one query above.

Output:

(3, 1536) (1536,)

resp.data holds one entry per input string, and resp.data[i] belongs to input[i]. That order is the only thing tying vectors to their documents, so if you sort, deduplicate, or filter the batch, carry the original positions along — one transposed batch shuffles the whole index and nothing reports an error.

Three rules follow from the shared space:

1. Queries and documents use the same model. Re-embedding half a corpus with a newer model leaves two coordinate systems in one collection. No exception is raised; the rankings simply stop agreeing.
2. Pin the model string. A provider can repoint an alias to new weights. Store the exact model name next to every vector and reject queries embedded with a different one.
3. Dimension is part of the contract. Widths of 1,536 and 3,072 do not share an index, and shrinking one to match the other is a deliberate quality trade, not a formatting fix.

Where dimension comes from, and what it costs, is the next chapter.

Every vector is written to an index, held in memory, and read back on each query, so width is a recurring cost rather than a one-time setting.

Newer models accept a dimensions argument that shortens each vector by dropping trailing components, with the training objective keeping the truncated form useful:

resp = client.embeddings.create(
    model='text-embedding-3-small',
    input=['reset my password'],
    dimensions=256,
)
print(len(resp.data[0].embedding))   # 256

Smaller is a trade, not an upgrade. Halving width halves index memory and speeds up probing, and the quality lost shows up first on hard queries — the ones where the right document sits just below the cutoff. Measure the exchange on your own queries (Evaluating Embedding Quality): four times smaller with six points of recall gone is a bad deal for a search feature and a fine one for a near-duplicate filter.

Failure modes at selection time:

• Mixed models in one collection. No error, incoherent rankings. Tag every vector with the model string that produced it.
• Dimension copied from a benchmark. Leaderboard corpora look nothing like your part numbers and ticket text.
• Newer assumed better. Retrieval-tuned models beat older generic ones on search, but one tuned for asymmetric search — short query against long document — is the wrong instrument for near-duplicate detection, where both sides are symmetric.
• Silent truncation at the input cap. Endpoints cap tokens per input, so a 40,000-character document loses its tail before reaching the model, and the clause that mattered was in the truncated part. Chunk long documents before embedding them.

One more property worth naming: the vector says nothing about why two texts are close. The model that placed get my money back near refund learned that connection from training data, and the same space will place genuinely unrelated texts near each other whenever they share style, length, or topic. Closeness is a starting signal for ranking, never evidence of relevance on its own — which is why every downstream decision about these vectors runs through a metric you choose deliberately, as Similarity Metrics for Vectors covers.