Skip to content

Semantic search#

This documentation applies to Piler enterprise edition releases 2.2.3+

Revision #1

Publication date: 2026-OCT-13

Semantic search finds emails by their meaning, not only by the words in them. A search for "budget approval" also finds the emails discussing it in other words.

Overview#

The scanner converts the subject and the body of every archived email to a vector (an embedding), and stores it in the Manticore <tenant id>_embeddings table. When a user runs a semantic search, the UI converts the search phrase the same way, looks up the closest emails, and combines them with the keyword search results (hybrid search).

Piler doesn't ship an embedding model. The embeddings are generated by an external OpenAI-compatible /v1/embeddings endpoint, typically a local Ollama instance, but LocalAI, vLLM, OpenAI, etc. work as well. The model is your choice, see Choosing a model below.

Things to know before you start:

  • Semantic search is disabled by default.
  • Both the scanner (embedding the emails) and the UI (embedding the search phrases) must use the same endpoint, model and prefixes.
  • The dimension of the model, SEMANTIC_SEARCH_DIMENSIONS and the knn_dims of the Manticore table must be the same number.
  • The email content only leaves your network if you point the endpoint at a cloud API. With a local Ollama instance everything stays on your infrastructure.

Setup#

1. Run an embedding endpoint#

Using Ollama on the host:

curl -fsSL https://ollama.com/install.sh | sh
ollama pull nomic-embed-text

Or using docker:

docker run -d --name ollama -v ollama:/root/.ollama -p 11434:11434 ollama/ollama
docker exec ollama ollama pull nomic-embed-text

Verify that the endpoint is reachable from the piler host, and check the dimension of the model:

curl -s http://localhost:11434/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"nomic-embed-text","input":"hello"}' | jq '.data[0].embedding | length'

It prints 768 for nomic-embed-text. This is the value of SEMANTIC_SEARCH_DIMENSIONS and knn_dims in the next steps.

Ollama doesn't need a GPU, though it's much faster with one:

Ollama (CPU only) Ollama (GPU)
Time per embedding ~30-80 ms ~2-5 ms
Throughput ~12-30 emails/s ~200-500 emails/s

CPU only is fine for a typical archive: 50,000 emails/day is ~35 emails/minute, and the one-time backfill of 1 million emails takes roughly 9 hours. The scanner generates the embeddings in the background, so its throughput doesn't affect the search speed.

2. Create the Manticore embeddings table#

The installer and the migrations create the embeddings table with knn_dims='384'. Unless your model has 384 dimensions, drop the table, and create it again with the dimension of your model. Replace TENANTID with piler in a no-tenant deployment, otherwise with the tenant id.

DROP TABLE IF EXISTS TENANTID_embeddings;
CREATE TABLE TENANTID_embeddings (
    id           bigint,
    participants text indexed,
    embedding    float_vector knn_type='hnsw' knn_dims='768' hnsw_similarity='cosine' hnsw_m='16' hnsw_ef_construction='200'
);

Run it on the Manticore SQL port, eg. mariadb -h0 -P9306

3. Configure piler#

Add the following to /var/piler/.env, it's read by both the UI and the scanner:

SEMANTIC_SEARCH_ENABLED=true
SEMANTIC_SEARCH_ENDPOINT_URL=http://localhost:11434/v1/embeddings
SEMANTIC_SEARCH_ENDPOINT_MODEL=nomic-embed-text
SEMANTIC_SEARCH_DIMENSIONS=768

Then restart the services:

systemctl restart piler-scanner piler-ui

When using docker compose, set the same variables for both the piler-ui and the scanner containers. Note that localhost inside a container is not the host, use the name of the ollama service instead:

services:
  piler-ui:
    image: sutoj/piler-ui:latest
    environment:
      SEMANTIC_SEARCH_ENABLED: "true"
      SEMANTIC_SEARCH_ENDPOINT_URL: "http://ollama:11434/v1/embeddings"
      SEMANTIC_SEARCH_ENDPOINT_MODEL: "nomic-embed-text"
      SEMANTIC_SEARCH_DIMENSIONS: "768"

  ollama:
    image: ollama/ollama
    volumes:
      - ollama_data:/root/.ollama

volumes:
  ollama_data:

4. Multi-tenant deployments#

The endpoint is configured once in the .env file as above, and it's shared by all tenants. In addition, semantic search must be enabled for each tenant in its settings:

UPDATE piler.tenant_settings
SET settings_json = JSON_SET(COALESCE(settings_json, '{}'), '$.semantic_search_enabled', true)
WHERE tenant_id = 'tenant1';

Set it to false to disable it for a tenant. Each enabled tenant gets its embeddings in its own <tenant id>_embeddings table.

5. Embed the already archived emails#

The scanner only processes the new emails. To generate the embeddings for the emails already in the archive, run the scanner once in backfill mode, with the same settings as the scanner service:

/usr/libexec/piler/scanner -once -backfill

6. Verify#

The scanner logs whether semantic search is enabled:

journalctl -u piler-scanner | grep -i semantic

✓ Semantic Search: ENABLED (dimensions: 768)

The embeddings table is filling up:

SELECT COUNT(*) FROM TENANTID_embeddings;

Config options#

Option Default Description
SEMANTIC_SEARCH_ENABLED false Enable semantic search: the scanner generates embeddings, and the UI offers the hybrid search.
SEMANTIC_SEARCH_ENDPOINT_URL "" URL of the OpenAI-compatible embeddings endpoint, eg. http://ollama:11434/v1/embeddings. Required.
SEMANTIC_SEARCH_ENDPOINT_MODEL "" The model name sent to the endpoint.
SEMANTIC_SEARCH_ENDPOINT_API_KEY "" Bearer token for the endpoint. Leave it empty for Ollama and other unauthenticated servers.
SEMANTIC_SEARCH_DIMENSIONS 384 The dimension of the embedding vectors. It must match the model and the knn_dims of the embeddings table.
SEMANTIC_SEARCH_DOC_PREFIX "" Text prepended to the email before embedding it. Only for asymmetric models, see below.
SEMANTIC_SEARCH_QUERY_PREFIX "" Text prepended to the search phrase before embedding it. Only for asymmetric models, see below.
SEMANTIC_SEARCH_MAX_TOKENS 1000 The maximum number of words of an email (subject + body) used for its embedding.
SEMANTIC_SEARCH_SCORE_THRESHOLD 0.65 Score threshold (0.0-1.0) of the semantic results. A lower value is stricter: more relevant, but fewer results.
SEMANTIC_SEARCH_KNN_EF 100 The query time ef parameter of the HNSW index. A higher value is more accurate, but slower.

Choosing a model#

Any embedding model served by your endpoint works. English-only models such as all-minilm are fast, but an archive with non-English emails needs a multilingual model.

Model Ollama name Dimensions Prefixes Notes
all-minilm all-minilm 384 No English only, fast
nomic-embed-text nomic-embed-text 768 No Good multilingual coverage, the easiest choice
multilingual-e5-base milkey/multilingual-e5-base 768 Yes 100+ languages, handles German compound words better, the best free option for Hebrew
multilingual-e5-small milkey/multilingual-e5-small 384 Yes Lighter, same languages, slightly lower quality
bge-m3 bge-m3 1024 No Multilingual, including non-Latin scripts
mxbai-embed-large mxbai-embed-large 1024 No Strong English quality
OpenAI text-embedding-3-small - 1536 No Excellent multilingual, cloud only

Recommendations by the language of the archive:

  • English: all-minilm, or mxbai-embed-large for better quality
  • Spanish, French: nomic-embed-text
  • German: multilingual-e5-base
  • Hebrew and other non-Latin scripts: multilingual-e5-base or bge-m3. If you'd rather not deal with prefixes, nomic-embed-text still handles Hebrew well.

An example for German with multilingual-e5-base:

SEMANTIC_SEARCH_ENABLED=true
SEMANTIC_SEARCH_ENDPOINT_URL=http://localhost:11434/v1/embeddings
SEMANTIC_SEARCH_ENDPOINT_MODEL=milkey/multilingual-e5-base
SEMANTIC_SEARCH_DIMENSIONS=768
SEMANTIC_SEARCH_DOC_PREFIX="passage: "
SEMANTIC_SEARCH_QUERY_PREFIX="query: "

An example for OpenAI:

SEMANTIC_SEARCH_ENABLED=true
SEMANTIC_SEARCH_ENDPOINT_URL=https://api.openai.com/v1/embeddings
SEMANTIC_SEARCH_ENDPOINT_MODEL=text-embedding-3-small
SEMANTIC_SEARCH_ENDPOINT_API_KEY=sk-...
SEMANTIC_SEARCH_DIMENSIONS=1536

Prefixes for asymmetric models#

Embedding models are either symmetric or asymmetric:

  • Symmetric models encode the emails and the search phrases the same way. Leave both prefixes empty.
  • Asymmetric models encode them differently, and need a prefix telling which one they get. With a wrong or missing prefix there's no error, the search results are just worse.
Model SEMANTIC_SEARCH_DOC_PREFIX SEMANTIC_SEARCH_QUERY_PREFIX
multilingual-e5-small/base/large "passage: " "query: "
e5-small/base/large "passage: " "query: "
EmbedGemma "Represent this document for retrieval: " "Query: "
all-minilm, nomic-embed-text, bge-m3 empty empty

The trailing space inside the quotes is required. Without it the model gets passage:text instead of passage: text, and the search quality degrades silently.

Changing the model or the prefixes#

Vectors of different models, or of the same model with different prefixes are not comparable. After changing the model or the prefixes:

  1. Drop and create the embeddings table with the new knn_dims, see step 2
  2. Update the .env file, and restart piler-scanner and piler-ui
  3. Run the scanner with -once -backfill to embed all emails again

Limitations#

  • Only the subject and the body of the emails are embedded, the attachments are not.
  • Users restricted to certain categories get keyword search only, and the UI doesn't show the semantic search option for them.
  • Storage: dimensions × 4 bytes per email, plus ~10-20% for the index. With 768 dimensions it's about 3 KB per email, ie. about 3 GB for 1 million emails.
  • If the endpoint is down, the search falls back to the keyword results, and the scanner logs the errors, and continues with its other tasks.

Troubleshooting#

SEMANTIC_SEARCH_ENDPOINT_URL is not configured

SEMANTIC_SEARCH_ENABLED=true, but the endpoint is not set. Set the URL for both the scanner and the UI.

Connection refused, timeouts

  • Check the endpoint from where piler runs, eg. inside the container (docker exec ... curl ...)
  • With Ollama, make sure the model is pulled: ollama pull nomic-embed-text

Dimension mismatch, Manticore insert errors

The dimension of the model, SEMANTIC_SEARCH_DIMENSIONS and the knn_dims of the table are not the same. See Changing the model or the prefixes.

No embeddings are generated

  1. Is semantic search enabled, in a multi-tenant deployment for the tenant as well?
  2. Is the scanner running? systemctl status piler-scanner
  3. Check the scanner logs for HTTP errors from the endpoint