How to Run EmbeddingGemma 2 With Ollama and Pick a Dimension

EmbeddingGemma 2 runs locally through Ollama and through sentence-transformers. Both paths give you 768-dimension vectors that you can cut down to 512, 256, or 128 dimensions with Matryoshka Representation Learning (MRL), which Google trained into the model. Ollama is the fastest way to get a vector out: pull the model, post text to the embeddings endpoint, store what comes back. sentence-transformers is the better fit when you want truncation handled at load time and you're already in Python.

The model itself has 740 million parameters, is built on the Gemma 4 architecture, and ships under Apache 2.0, so commercial use is fine. Text-only workloads need as little as 270M of those parameters, and the context window is 8K tokens. It also embeds images, audio and video, but the Ollama and sentence-transformers setup below is about the text and code side, which is what most retrieval pipelines need.

What EmbeddingGemma 2 gives you over the first version#

The first EmbeddingGemma was text-only. Version 2 keeps the same multilingual text quality and adds a shared embedding space for code, images, video and audio. The part that matters most for developers is code: Google reports a 9.92-point improvement on MTEB Code, from 68.76 to 78.68. That's the number to care about if you're indexing a repository for semantic search or feeding retrieval into a coding agent.

Three other properties shape how you'll deploy it:

• Modular weights. Text-only needs about 270M parameters. The vision encoder adds 170M and the audio encoder adds 300M. If you only embed text and code, you don't pay for the rest.
• Matryoshka output. The native vector is 768 dimensions. You can keep the first 512, 256 or 128 values and still get a usable embedding, which Google describes as up to 6x storage reduction for a local vector database.
• 8K context. Four times the first version. Long documents and whole source files fit in one pass instead of being split into tiny chunks.

Full metrics and the per-modality details live in the model card on Hugging Face. Google's Gemma documentation is the place to check for current guides on inference and fine-tuning.

Pull and run EmbeddingGemma 2 with Ollama#

Ollama is one of the launch partners, so the model is available from the Ollama library. Tags do change, so confirm the exact name on the library page before you pull. The commands below use a shell variable so you only type the tag once.

MODEL=embeddinggemma2   # replace with the tag shown in the Ollama library
ollama pull "$MODEL"
ollama list

Embedding models don't chat, so ollama run isn't the useful command here. You talk to the local HTTP server instead. Ollama listens on port 11434 by default and exposes an embeddings endpoint that takes a model name and an input string or list of strings:

curl -s http://localhost:11434/api/embed \
  -d "{\"model\": \"$MODEL\", \"input\": \"how do I rotate a JWT signing key\"}"

The response contains the vector as a JSON array of floats. The exact field names and any optional parameters are documented in the Ollama API reference, and that's where to look if a newer server version adds server-side truncation or batching options.

From Python, the standard library is enough. This isn't a run-in-the-browser example because it needs the Ollama server:

import json
import urllib.request

MODEL = "embeddinggemma2"  # same tag you pulled

def embed(texts):
    body = json.dumps({"model": MODEL, "input": texts}).encode()
    req = urllib.request.Request(
        "http://localhost:11434/api/embed",
        data=body,
        headers={"Content-Type": "application/json"},
    )
    with urllib.request.urlopen(req) as resp:
        return json.load(resp)["embeddings"]

vectors = embed(["rotate a signing key", "def rotate_key(kid): ..."])
print(len(vectors), len(vectors[0]))

You should see two vectors of 768 values. Everything else in a RAG pipeline, including chunking, storage and nearest-neighbor search, is independent of the model and works the same as it did with any other Ollama embedding model.

Generate embeddings with sentence-transformers#

If you'd rather load the weights directly, sentence-transformers is the supported Python path. Install it from PyPI, then load the model by its Hugging Face ID. If the model page asks you to accept terms before download, do that first with your Hugging Face account.

from sentence_transformers import SentenceTransformer

MODEL_ID = "google/..."  # copy the exact ID from the Hugging Face model card

model = SentenceTransformer(MODEL_ID)

docs = [
    "Rotate the JWT signing key every 90 days.",
    "def rotate_key(kid: str) -> None: ...",
]
emb = model.encode(docs)
print(emb.shape)  # (2, 768)

To get a shorter vector, pass truncate_dim when you load the model. sentence-transformers then slices and normalizes for you, so every call to encode returns the shorter vector:

model = SentenceTransformer(MODEL_ID, truncate_dim=256)
emb = model.encode(docs)
print(emb.shape)  # (2, 256)

query = model.encode("how often should I rotate keys")
print(model.similarity(query, emb))

One thing to check on the model card rather than assume: Gemma embedding models have typically used distinct prompts for queries and documents. The card lists the prompt names it expects, and sentence-transformers can apply them automatically when the model's config includes them. Read the card, use what it says, and don't copy prompt strings from a post about the first EmbeddingGemma.

Picking 768, 512, 256 or 128 dimensions with Matryoshka truncation#

Matryoshka Representation Learning trains the model so that the most important information sits in the leading dimensions. That's why you can drop the tail and still retrieve well. The trade is simple: fewer dimensions means smaller storage, faster distance computations, and some loss in ranking quality that grows as you cut deeper.

My advice on where to start:

• 768 when storage isn't a concern and you want the best ranking. Fine for a few hundred thousand chunks on a laptop.
• 512 as a default for most local apps. The saving is modest but real, and quality stays close to full size.
• 256 when you're on a phone, an edge box, or you're embedding millions of chunks into SQLite or a small vector store.
• 128 when memory is the hard limit. Expect to re-rank the top results with something else.

The mechanics of truncation are the same whether the model, the library or your own code does it: keep the first N values, then normalize the shortened vector back to unit length so cosine similarity still behaves. Here's that logic with stand-in vectors so you can see the shape of the operation. The similarity values come from random data, not the model, so they say nothing about EmbeddingGemma 2's quality at each size.

import math
import random

random.seed(7)

def unit(v):
    n = math.sqrt(sum(x * x for x in v))
    return [x / n for x in v]

def cosine(a, b):
    return sum(x * y for x, y in zip(a, b))

def truncate(v, dim):
    # Matryoshka: keep the leading dims, then re-normalize
    return unit(v[:dim])

# Stand-ins for two 768-d embeddings; real ones come from the model
doc = unit([random.gauss(0, 1) for _ in range(768)])
query = unit([d + random.gauss(0, 0.4) for d in doc])

for dim in (768, 512, 256, 128):
    raw_norm = math.sqrt(sum(x * x for x in doc[:dim]))
    a, b = truncate(doc, dim), truncate(query, dim)
    print(f"{dim:>3} dims  norm before renormalize={raw_norm:.3f}  cosine={cosine(a, b):.4f}")

The norm before renormalize column is the point. A sliced vector is no longer unit length, and if you skip the renormalize step your cosine scores drift and your threshold tuning breaks.

Storage is easier to reason about with numbers. Each float32 is 4 bytes, so a corpus of one million chunks costs this much at each size:

const dims = [768, 512, 256, 128];
const chunks = 1_000_000;
const bytesPerFloat = 4; // float32

for (const d of dims) {
  const gib = (chunks * d * bytesPerFloat) / 1024 ** 3;
  const factor = 768 / d;
  console.log(`${String(d).padStart(3)} dims: ${gib.toFixed(2)} GiB for ${chunks.toLocaleString('en-US')} vectors (${factor}x smaller than 768)`);
}

Index overhead on top of the raw floats depends on your vector store, but the ratio between sizes holds.

Embedding code with EmbeddingGemma 2#

The code improvement is the headline feature for this release, and it changes what you can do locally. A few practical notes for codebase indexing:

• Chunk by structure, not by line count. Functions, classes and top-level blocks make better units than fixed windows. With an 8K context you can often embed a whole file or a large function in one go, so don't split more than you have to.
• Embed natural-language queries against code documents. The shared embedding space is built so that a question like "where do we validate the upload size" lands near the function that does it. Use the query prompt for the question and the document prompt for the code, as the model card specifies.
• Keep file path and language as metadata, not in the vector. Filtering by path or language in your store is cheaper and more precise than hoping the embedding encodes it.
• Pair it with a local generator. EmbeddingGemma 2 shares its text tokenizer with Gemma 4, so running both through Ollama for retrieval plus generation keeps the combined memory footprint lower than mixing unrelated model families.

Common mistakes and gotchas#

Pulling the wrong tag. Ollama library tags for embedding models have changed between releases, and quantization variants show up under separate tags. Check the library page, not a snippet from a forum, and pin the tag you tested in your config.

Mixing dimensions in one index. If you embed half your corpus at 768 and the rest at 256, nearest-neighbor search is meaningless. Decide the dimension once, write it next to the model tag in your config, and reindex everything if you change either.

Truncating without renormalizing. Covered above, but it's the most common bug in hand-rolled truncation. sentence-transformers' truncate_dim handles it. If you slice Ollama output yourself, normalize afterward.

Using chat-model prompts. The embeddings endpoint ignores system prompts and chat templates. The task prompts that matter are the ones the model card documents for queries and documents.

Comparing vectors from different models or versions. EmbeddingGemma 2 vectors aren't compatible with EmbeddingGemma 1 vectors. Upgrading means a full reindex.

Expecting multimodal input through Ollama's text endpoint. The model can embed images, audio and video, but the paths covered here are text and code. For multimodal on-device work, Google points to its AI Edge tooling, LiteRT and MediaPipe. Check the model card and Ollama docs for what each runtime supports today.

Oversized chunks. 8K tokens is generous, but anything beyond it gets cut. Measure your longest chunk against the tokenizer instead of assuming.

Quick reference#

Checklist before you ship:

1. Pin the Ollama tag or Hugging Face model ID in config.
2. Pin the dimension in the same config and store it alongside the index.
3. Use the query and document prompts from the model card.
4. Normalize after any client-side truncation.
5. Reindex everything on any model or dimension change.
6. Re-read the Hugging Face model card and Ollama docs before upgrading, since prompts and tags move between releases.

Quick answers