hwdsl2/embeddings-server repository overview

⁠EmbedCrate

Open-source, self-hosted text embeddings and reranking API.

GitHub: https://github.com/hwdsl2/embedcrate⁠

Run text embeddings and reranking on your own server with EmbedCrate. Powered by Hugging Face Text Embeddings Inference (TEI)⁠, it provides an OpenAI-compatible /v1/embeddings API and an optional /rerank endpoint, with configurable models and persistent caching.

Features:

• OpenAI-compatible API: POST /v1/embeddings for embedding requests from compatible OpenAI SDKs and apps.
• Private, local processing: text data stays on your server and is not sent to third parties.
• Reranking: use POST /rerank with a cross-encoder model to re-score retrieved documents for higher retrieval accuracy.
• Embedding models: choose from BAAI/bge-small-en-v1.5, BAAI/bge-m3, nomic-embed-text-v1.5, and more.
• High-performance backend: powered by Hugging Face TEI⁠, a high-performance embeddings server written in Rust.
• Offline operation: run without internet access using pre-cached models (EMBED_LOCAL_ONLY).
• Automated builds: images are automatically built and published through GitHub Actions⁠.

Also available as part of the Self-Hosted AI Stack⁠, which deploys a complete self-hosted AI stack with a single command.

Also available:

• Related AI services: ScribeCrate⁠, SpeakCrate⁠, GatewayCrate⁠, InferCrate⁠, ParseCrate⁠, UplinkCrate⁠

⁠Quick start

Use this command to set up a text embeddings server:

docker run \
    --name embeddings \
    --restart=always \
    -v embeddings-data:/var/lib/embeddings \
    -p 8000:8000 \
    -d hwdsl2/embeddings-server

Note: For internet-facing deployments, use a reverse proxy⁠ to add HTTPS. Also replace -p 8000:8000 with -p 127.0.0.1:8000:8000 in the docker run command above, to prevent direct access to the unencrypted port.

The default model BAAI/bge-small-en-v1.5 (~130 MB) is downloaded and cached on first start. Check the logs to confirm the server is ready:

docker logs embeddings

Once you see "Text embeddings server is ready", generate your first embeddings:

Fresh persistent installations require an API key. Retrieve it for the following examples:

embed_api_key="$(docker exec embeddings embed_manage --getkey)"

curl http://your_server_ip:8000/v1/embeddings \
    -H "Authorization: Bearer $embed_api_key" \
    -H "Content-Type: application/json" \
    -d '{"input": "The quick brown fox", "model": "text-embedding-ada-002"}'

Response:

{"object":"list","data":[{"object":"embedding","embedding":[0.032,...,-0.017],"index":0}],"model":"BAAI/bge-small-en-v1.5","usage":{"prompt_tokens":5,"total_tokens":5}}

⁠Community

• 📬 Get project updates and free deployment guides⁠ (1–2 emails/month)
• 💬 Join the r/selfhostedstack⁠ community for discussions and showcases
• ⭐ Star the repository if you find it useful — it helps others discover it

⁠Requirements

• A Linux server (local or cloud) with Docker installed
• Supported architectures: amd64 (x86_64), arm64 (aarch64, e.g. AWS Graviton, Apple Silicon VMs)
• Minimum RAM: ~250 MB free for the default BAAI/bge-small-en-v1.5 model (see model table⁠)
• Internet access for the initial model download (the model is cached locally afterwards). Not required if using EMBED_LOCAL_ONLY=true with pre-cached models.

For internet-facing deployments, see Using a reverse proxy⁠ to add HTTPS.

⁠Download

Get the trusted build from the Docker Hub registry⁠:

docker pull hwdsl2/embeddings-server

Alternatively, you may download from Quay.io⁠:

docker pull quay.io/hwdsl2/embeddings-server
docker image tag quay.io/hwdsl2/embeddings-server hwdsl2/embeddings-server

Supported platforms: linux/amd64, linux/arm64.

⁠Environment variables

All variables are optional. Fresh installs with a mounted /var/lib/embeddings volume auto-generate a Bearer token. Existing installs without a key remain open for backward compatibility.

This Docker image uses the following variables, that can be declared in an env file (see example⁠):

Note: In your env file, you may enclose values in single quotes, e.g. VAR='value'. Do not add spaces around =. If you change EMBED_PORT, update the -p flag in the docker run command accordingly.

Example using an env file:

cp embed.env.example embed.env
# Edit embed.env with your settings, then:
docker run \
    --name embeddings \
    --restart=always \
    -v embeddings-data:/var/lib/embeddings \
    -v ./embed.env:/embed.env:ro \
    -p 8000:8000 \
    -d hwdsl2/embeddings-server

The env file is bind-mounted into the container, so changes are picked up on every restart without recreating the container.

docker run \
    --name embeddings \
    --restart=always \
    -v embeddings-data:/var/lib/embeddings \
    -p 8000:8000 \
    --env-file=embed.env \
    -d hwdsl2/embeddings-server

⁠Using docker-compose

cp embed.env.example embed.env
# Edit embed.env as needed, then:
docker compose up -d
docker logs embeddings

Example docker-compose.yml (already included):

services:
  embeddings:
    image: hwdsl2/embeddings-server
    container_name: embeddings
    restart: always
    ports:
      - "8000:8000/tcp"  # For a host-based reverse proxy, change to "127.0.0.1:8000:8000/tcp"
      # - "8001:8001/tcp"  # Reranker API (uncomment if RERANK_ENABLED=true in embed.env)
    volumes:
      - embeddings-data:/var/lib/embeddings
      - ./embed.env:/embed.env:ro

volumes:
  embeddings-data:
    name: embeddings-data

Note: For internet-facing deployments, use a reverse proxy⁠ to add HTTPS. Also change "8000:8000/tcp" to "127.0.0.1:8000:8000/tcp" in docker-compose.yml, to prevent direct access to the unencrypted port.

⁠API reference

The API is compatible with OpenAI's embeddings endpoint⁠. For clients using the OpenAI SDK, configure the base URL and your server's API key:

The /v1/embeddings endpoint is served directly by TEI. Supported OpenAI request fields depend on TEI; fields such as encoding_format, dimensions, user, and token-array inputs are upstream-dependent and not documented or tested by this image.

Fresh persistent installations require an API key. Retrieve it for the following examples:

embed_api_key="$(docker exec embeddings embed_manage --getkey)"

export OPENAI_BASE_URL="http://your_server_ip:8000"
export OPENAI_API_KEY="$embed_api_key"

If API key authentication is disabled, omit the Authorization header in curl examples. OpenAI SDK clients still require a nonempty key; set OPENAI_API_KEY=unused.

⁠Generate embeddings

POST /v1/embeddings
Content-Type: application/json

Parameters:

Example — single input:

curl http://your_server_ip:8000/v1/embeddings \
    -H "Authorization: Bearer $embed_api_key" \
    -H "Content-Type: application/json" \
    -d '{"input": "The quick brown fox", "model": "text-embedding-ada-002"}'

Example — batch input:

curl http://your_server_ip:8000/v1/embeddings \
    -H "Authorization: Bearer $embed_api_key" \
    -H "Content-Type: application/json" \
    -d '{"input": ["First sentence", "Second sentence"], "model": "text-embedding-ada-002"}'

With API key authentication:

curl http://your_server_ip:8000/v1/embeddings \
    -H "Authorization: Bearer $embed_api_key" \
    -H "Content-Type: application/json" \
    -d '{"input": "Your text here", "model": "text-embedding-ada-002"}'

Response:

{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "embedding": [0.032, -0.018, ...],
      "index": 0
    }
  ],
  "model": "BAAI/bge-small-en-v1.5",
  "usage": { "prompt_tokens": 5, "total_tokens": 5 }
}

⁠Model info

GET /info

Returns the active model ID, maximum input length, and server version.

curl http://your_server_ip:8000/info \
    -H "Authorization: Bearer $embed_api_key"

⁠Rerank documents

POST /rerank
Content-Type: application/json

Parameters:

Example:

By default, the reranker uses the embeddings key. If you configured a separate RERANK_API_KEY, replace the value below with that key. If reranker authentication is disabled, omit the Authorization header.

rerank_api_key="$embed_api_key"

curl http://your_server_ip:8001/rerank \
    -H "Authorization: Bearer $rerank_api_key" \
    -H "Content-Type: application/json" \
    -d '{
      "query": "What is deep learning?",
      "texts": [
        "Deep learning is a subset of machine learning...",
        "The weather today is sunny with a high of 75°F.",
        "Neural networks are inspired by the human brain."
      ],
      "raw_scores": false
    }'

Response:

[
  {"index": 0, "score": 0.98},
  {"index": 2, "score": 0.72},
  {"index": 1, "score": 0.01}
]

Results are sorted by relevance score (highest first). Use this to re-rank documents retrieved by embeddings similarity search.

⁠Interactive API docs

An interactive Swagger UI is available at:

http://your_server_ip:8000/docs

If reranking is enabled, the reranker also has its own interactive docs at:

http://your_server_ip:8001/docs

⁠Persistent data

All server data is stored in the Docker volume (/var/lib/embeddings inside the container):

/var/lib/embeddings/
├── models--BAAI--bge-small-en-v1.5/   # Cached embedding model files
├── models--BAAI--bge-reranker-v2-m3/  # Cached reranker model files (if enabled)
├── .port                # Active port (used by embed_manage)
├── .model               # Active model ID (used by embed_manage)
├── .rerank_model        # Active reranker model (used by embed_manage)
├── .rerank_port         # Active reranker port (used by embed_manage)
└── .server_addr         # Cached server IP (used by embed_manage)

Back up the Docker volume to preserve downloaded models. Models range from ~90 MB to ~1.3 GB and are only downloaded once; preserving the volume avoids re-downloading on container recreation.

⁠Managing the server

Use embed_manage inside the running container to inspect and manage the server.

Show server info:

docker exec embeddings embed_manage --showinfo

List recommended models:

docker exec embeddings embed_manage --listmodels

List recommended reranker models:

docker exec embeddings embed_manage --listrerankers

Pre-download a model:

docker exec embeddings embed_manage --pullmodel BAAI/bge-base-en-v1.5
docker exec embeddings embed_manage --pullmodel BAAI/bge-reranker-v2-m3

⁠Switching the model

To change the active model:

1. (Optional but recommended) Pre-download the new model while the server is running:

   docker exec embeddings embed_manage --pullmodel BAAI/bge-base-en-v1.5
   

   Copy

2. Update EMBED_MODEL in your embed.env file (or add -e EMBED_MODEL=BAAI/bge-base-en-v1.5 to your docker run command).

3. Restart the container:

   docker restart embeddings
   

   Copy

Recommended models:

Models are cached in the /var/lib/embeddings Docker volume and only downloaded once. Any HuggingFace model supported by TEI can be used — see the TEI supported models list⁠.

⁠Reranking

Reranking improves retrieval quality by re-scoring documents with a cross-encoder model. Enable it by setting RERANK_ENABLED=true in your env file.

⁠Quick setup

1. Add to your embed.env:

   RERANK_ENABLED=true
   

   Copy

2. Expose port 8001 (add -p 8001:8001 to your docker run command, or uncomment the port in docker-compose.yml).

3. Restart the container:

   docker restart embeddings
   

   Copy

The reranker model (BAAI/bge-reranker-v2-m3, ~560 MB) is downloaded on first start.

⁠Operating modes

In rerank-only mode, the reranker listens on port 8000 by default (since the embeddings process is disabled), unless RERANK_PORT is explicitly set.

⁠Recommended reranker models

⁠Using with GatewayCrate

To use the reranker with LiteLLM, including GatewayCrate⁠, add it as a rerank model in your LiteLLM config:

model_list:
  - model_name: rerank
    litellm_params:
      model: huggingface/BAAI/bge-reranker-v2-m3
      api_base: http://embeddings:8001
      api_key: os.environ/RERANK_API_KEY

Set RERANK_API_KEY in the LiteLLM container environment to the key accepted by your reranker. By default, this is the embeddings key retrieved above; use your separately configured reranker key if applicable. If reranker authentication is disabled, omit the api_key entry.

Then call the LiteLLM /rerank endpoint, and it will proxy to your self-hosted reranker.

⁠Securing your server

See Securing your server⁠.

⁠Using a reverse proxy

For internet-facing deployments, place a reverse proxy in front of the embeddings server to handle HTTPS termination. The server works without HTTPS on a local or trusted network, but HTTPS is recommended when the API endpoint is exposed to the internet.

Use one of the following addresses to reach the embeddings container from your reverse proxy:

• embeddings:8000 — if your reverse proxy runs as a container in the same Docker network as the embeddings server (e.g. defined in the same docker-compose.yml).
• 127.0.0.1:8000 — if your reverse proxy runs on the host and port 8000 is published (the default docker-compose.yml publishes it).

Example with Caddy⁠ (Docker image⁠) (automatic TLS via Let's Encrypt, reverse proxy in the same Docker network):

Caddyfile:

embeddings.example.com {
  reverse_proxy embeddings:8000
}

Example with nginx (reverse proxy on the host):

server {
    listen 443 ssl;
    server_name embeddings.example.com;

    ssl_certificate     /path/to/cert.pem;
    ssl_certificate_key /path/to/key.pem;

    location / {
        proxy_pass         http://127.0.0.1:8000;
        proxy_set_header   Host $host;
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto $scheme;
        proxy_read_timeout 120s;
    }
}

Fresh persistent installs auto-generate an EMBED_API_KEY. Display it with docker exec embeddings embed_manage --showkey, or use docker exec embeddings embed_manage --getkey in scripts. For existing installs without a key, set EMBED_API_KEY in your env file to enable authentication.

⁠Update Docker image

To update the Docker image and container, first download⁠ the latest version:

docker pull hwdsl2/embeddings-server

If the Docker image is already up to date, you should see:

Status: Image is up to date for hwdsl2/embeddings-server:latest

Otherwise, it will download the latest version. Remove and re-create the container:

docker rm -f embeddings
# Then re-run the docker run command from Quick start with the same volume and port.

Your downloaded models are preserved in the embeddings-data volume.

⁠Using with other AI services

Embeddings can be used as the embedding service in a broader self-hosted AI setup.

For full and lightweight Docker Compose stacks, manual docker run examples, and voice/RAG/MCP pipeline examples with SpeakCrate, EmbedCrate, GatewayCrate, InferCrate, ParseCrate, and UplinkCrate, see Self-Hosted AI Stack⁠.

⁠Usage counts

See Usage counts⁠.

⁠Technical details

• Base image (amd64): ghcr.io/huggingface/text-embeddings-inference:cpu-latest (Debian)
• Base image (arm64): Built from TEI source with ONNX Runtime + Candle backends (Debian)
• Embeddings engine: Hugging Face TEI⁠ (Rust-based, high-performance)
• API: OpenAI-compatible /v1/embeddings endpoint (served directly by TEI; supported fields depend on TEI)
• Reranking: TEI /rerank endpoint via a second process loaded with a cross-encoder model
• Data directory: /var/lib/embeddings (Docker volume)
• Model storage: HuggingFace Hub format inside the volume — downloaded once, reused on restarts
• Model management: Python (huggingface_hub) for pre-download via embed_manage --pullmodel

⁠License

Note: The software components inside the pre-built image (such as Hugging Face TEI and its dependencies) are under the respective licenses chosen by their respective copyright holders. As for any pre-built image usage, it is the image user's responsibility to ensure that any use of this image complies with any relevant licenses for all software contained within.

Copyright (C) 2026 Lin Song
This work is licensed under the MIT License⁠.

Hugging Face Text Embeddings Inference (TEI) is Copyright (C) Hugging Face, Inc., and is distributed under the Apache License 2.0⁠.

This project is an independent Docker setup for Hugging Face TEI and is not affiliated with, endorsed by, or sponsored by Hugging Face, Inc.