Context Engine configuration reference

How configuration works

ContextEngineConfig is the one settings object every part of the engine reads. You build it once and pass it to ContextEngine. Both packages have the same sections and the same defaults, with snake_case names in Python and camelCase names in TypeScript.

Loading from the environment

Every field can come from an environment variable: prefix CE_, the field name in upper case, and __ between nesting levels. embedding.api_key is CE_EMBEDDING__API_KEY, and graph.extraction_llm.model is CE_GRAPH__EXTRACTION_LLM__MODEL. The same variable names work in both packages; TypeScript converts each segment to camelCase.

from context_engine import ContextEngine, ContextEngineConfig

# No arguments: every field comes from CE_* environment variables.
config = ContextEngineConfig()

# Keyword arguments win over the environment, field by field.
config = ContextEngineConfig(database_url="postgresql://user:pass@db:5432/app")

engine = ContextEngine(config)

import { ContextEngine, ContextEngineConfig } from "@promptev/context-engine";

// Reads CE_* from process.env. Throws unless CE_DATABASE_URL and
// CE_EMBEDDING__PROVIDER / CE_EMBEDDING__MODEL (or overrides) are present.
const config = ContextEngineConfig.fromEnv();

// Overrides are deep-merged over the environment.
const withOverride = ContextEngineConfig.fromEnv({
  databaseUrl: "postgresql://user:pass@db:5432/app",
});

const engine = new ContextEngine(config);

CE_DATABASE_URL=postgresql://user:pass@localhost:5432/mydb
CE_EMBEDDING__PROVIDER=openai
CE_EMBEDDING__MODEL=text-embedding-3-small
CE_EMBEDDING__API_KEY=sk-...
CE_LLM__PROVIDER=openai
CE_LLM__MODEL=gpt-4o-mini
CE_LLM__API_KEY=sk-...
CE_GRAPH__ENABLED=true
CE_GRAPH__EXTRACTION_LLM__PROVIDER=openai
CE_GRAPH__EXTRACTION_LLM__MODEL=gpt-4o-mini
CE_GRAPH__EXTRACTION_LLM__API_KEY=sk-...
CE_INGEST__MAX_ROWS=500000
CE_SECRET_KEY=<base64url 32-byte key>

Full example

Most sections at once. Anything you leave out keeps the default in the tables below.

from context_engine import (
    ContextEngine, ContextEngineConfig, EmbeddingConfig, LLMConfig,
    GraphConfig, RerankerConfig, SearchConfig, StorageConfig,
)
from context_engine.config import IngestConfig

config = ContextEngineConfig(
    database_url="postgresql://user:pass@localhost:5432/mydb",
    embedding=EmbeddingConfig(provider="openai", model="text-embedding-3-small",
                              api_key="sk-...", max_retries=1),
    llm=LLMConfig(provider="openai", model="gpt-4o-mini", api_key="sk-..."),
    vision_llm=LLMConfig(provider="openai", model="gpt-4o-mini", api_key="sk-..."),
    graph=GraphConfig(
        enabled=True,  # Postgres-only graph mode: no neo4j_uri
        extraction_llm=LLMConfig(provider="openai", model="gpt-4o-mini", api_key="sk-..."),
        rerank_weights={"entity_match": 0.4},  # the other four keep their defaults
    ),
    reranker=RerankerConfig(candidates=50),  # the window a host reranker= sees
    search=SearchConfig(vector_floor="adaptive"),
    ingest=IngestConfig(max_rows=500_000),
    storage=StorageConfig(pool_size=10, max_overflow=20, pool_recycle=300),
    secret_key="...",  # base64url, 32 bytes
)
engine = ContextEngine(config)

import { ContextEngine, ContextEngineConfig } from "@promptev/context-engine";

const config = new ContextEngineConfig({
  databaseUrl: "postgresql://user:pass@localhost:5432/mydb",
  embedding: { provider: "openai", model: "text-embedding-3-small",
               apiKey: "sk-...", maxRetries: 1 },
  llm: { provider: "openai", model: "gpt-4o-mini", apiKey: "sk-..." },
  visionLlm: { provider: "openai", model: "gpt-4o-mini", apiKey: "sk-..." },
  graph: {
    enabled: true, // Postgres-only graph mode: no neo4jUri
    extractionLlm: { provider: "openai", model: "gpt-4o-mini", apiKey: "sk-..." },
    rerankWeights: { entity_match: 0.4 }, // the other four keep their defaults
  },
  reranker: { candidates: 50 }, // the window a host reranker sees
  search: { vectorFloor: "adaptive" },
  ingest: { maxRows: 500_000 },
  storage: { poolMax: 20, poolIdleTimeoutMs: 30_000 },
  secretKey: "...", // base64url, 32 bytes
});
const engine = new ContextEngine(config);

Top-level fields

Fields that sit directly on ContextEngineConfig.

Embedding

EmbeddingConfig, required. Environment prefix CE_EMBEDDING__.

Provider request limits

The defaults used when max_batch_items, max_batch_tokens or max_input_tokens is left unset. Set one to override it, for example against a self-hosted server with different limits.

LLM

LLMConfig is used in three places, each with its own environment prefix: llm (CE_LLM__), vision_llm (CE_VISION_LLM__) and graph.extraction_llm (CE_GRAPH__EXTRACTION_LLM__). The table shows the llm names. All three are optional when the engine is given a host model (see Bring your own model), which then makes every model call.

Providers

The built-in client is one OpenAI-compatible client over plain HTTP (POST {base_url}/chat/completions, POST {base_url}/embeddings); no provider SDK is installed. What each provider value reads from its config. Provider names are the same in both packages. For any other model or embedding API, pass model / embedder to the engine (see Bring your own model).

provider names the wire format, not the vendor. anthropic, gemini, vertex_ai and bedrock are refused at construction with a message naming the way forward. Every vendor still works, through a host model or through custom with its OpenAI-compatible root: the per-vendor table is under Use any model provider.

Graph

GraphConfig, off by default. Environment prefix CE_GRAPH__. Migrate with context-engine migrate --graph before turning it on. Python needs the [graph] extra; TypeScript needs graphology and graphology-communities-louvain, plus neo4j-driver only with a Neo4j.

To ask whether the Neo4j backend is configured, use config.graph.neo4j_configuredgraphNeo4jConfigured(config.graph).

Reranker

Reranking is bring-your-own: ContextEngine(config, reranker=fn) / new ContextEngine(config, { reranker }) with fn(query, docs) returning indices into docs, most relevant first, sync or async. There is no built-in rerank provider. RerankerConfig (environment prefix CE_RERANKER__) holds only the window; naming a provider field on it raises at construction. The engine drops out-of-range and duplicate indices and appends anything omitted, so a reranker can only reorder hits; one that raises ends that search on the fused order. Hits are masked by the redaction policy before the reranker sees them.

Fusion

FusionConfig. Environment prefix CE_FUSION__.

Search

SearchConfig: how much retrieval reacts to the shape of the query. Environment prefix CE_SEARCH__. With both fields unset, retrieval uses no vector floor and the query-shaped trigram threshold.

Extraction

ExtractionConfig. Environment prefix CE_EXTRACTION__. Tune the defaults against your own documents.

Ingest

IngestConfig. Environment prefix CE_INGEST__. The intake caps apply before a document is read or parsed, and they never sample or truncate: over a cap is a refusal.

Storage and pools

StorageConfig. Environment prefix CE_STORAGE__. The pool fields differ by port because they are passed straight through: to sqlalchemy.create_engine in Python and to pg.Pool in TypeScript. Size them for your workers times their concurrent searches, against what the database allows.

Redaction

redaction takes a RedactionPolicy: a list of rules plus any custom detector functions they reference. An empty policy, the default, does nothing. It holds functions, so it is set in code rather than from the environment.

from context_engine import ContextEngineConfig, RedactionPolicy, RedactionRule

config = ContextEngineConfig(
    # ... database_url, embedding ...
    redaction=RedactionPolicy(rules=[
        RedactionRule(name="email", detector="email"),                    # mask on output
        RedactionRule(name="ssn", detector="ssn", apply_at="both"),
        RedactionRule(name="customer", pattern=r"CUST-\d{6}", action="hash"),
    ]),
    redaction_secret_key="...",  # used by the hash rule; falls back to secret_key
)

import { ContextEngineConfig, RedactionPolicy } from "@promptev/context-engine";

const config = new ContextEngineConfig({
  // ... databaseUrl, embedding ...
  redaction: new RedactionPolicy({
    rules: [
      { name: "email", detector: "email" },                 // mask on output
      { name: "ssn", detector: "ssn", applyAt: "both" },
      { name: "customer", pattern: "CUST-\\d{6}", action: "hash" },
    ],
  }),
  redactionSecretKey: "...", // used by the hash rule; falls back to secretKey
});

Policy fields

rules (a list of RedactionRule) and custom_detectorscustomDetectors, a map from detector name to a function that takes text and returns (start, end) spans. Rule names must be unique, and every detector must be built in or present in the custom map.

Rule fields

Presidio detectors

Microsoft Presidio can supply detectors for the custom map. The two ports do it differently:

Secret keys

Two keys, protecting two different things. Neither is required until a feature needs it.

One way to generate a secret_key that both packages accept:

python -c "import base64, secrets; print(base64.urlsafe_b64encode(secrets.token_bytes(32)).decode())"

Secret fields

Every field that holds a credential is a secret type on the built config, in both packages: database_url, secret_key, redaction_secret_key, every api_key (embedding, llm, vision_llm, graph.extraction_llm) and neo4j_password. You pass plain strings, in code or through the CE_ variables; the config wraps them, and the value is masked in logs and serialisation.

from context_engine import ContextEngineConfig, EmbeddingConfig

config = ContextEngineConfig(
    database_url="postgresql://user:pass@localhost:5432/mydb",   # plain strings in
    embedding=EmbeddingConfig(provider="openai", model="text-embedding-3-small", api_key="sk-..."),
)

print(config.embedding.api_key)                     # **********
config.embedding.api_key.get_secret_value()         # "sk-..."
config.model_dump()["database_url"]                 # SecretStr('**********')
config.embedding.api_key == "sk-..."                # False: compare the read value

import { ContextEngineConfig, secretValue } from "@promptev/context-engine";

const config = new ContextEngineConfig({
  databaseUrl: "postgresql://user:pass@localhost:5432/mydb", // plain strings in
  embedding: { provider: "openai", model: "text-embedding-3-small", apiKey: "sk-..." },
});

console.log(config.embedding.apiKey);   // **********
JSON.stringify(config);                 // every secret field is "**********"
config.embedding.apiKey?.get();         // "sk-..."
secretValue(config.embedding.apiKey);   // "sk-...", also takes a plain string or null

Validation

These combinations raise when the config is constructed (ValueError in Python, Error in TypeScript). TypeScript messages use the camelCase field names.

Field bounds are enforced too: for example max_retries below 0, safety_margin above 1, or max_file_bytes of 0 are rejected. Redaction rules are checked when they are built: exactly one of detector / pattern, a valid regex, no field, and no unless on an ingest-only rule.

A config error never echoes its input, in either package, because the input is where the keys and passwords live. In Python the message leaves it out, but pydantic’s err.errors() still returns the raw input by design, so log err.errors(include_input=False):

import logging

from pydantic import ValidationError

from context_engine import ContextEngineConfig

log = logging.getLogger(__name__)

try:
    config = ContextEngineConfig()
except ValidationError as err:
    log.error("bad config: %s", err.errors(include_input=False))   # never the raw input

Other environment variables

Variables outside the ContextEngineConfig scheme that the code reads.

The CLI’s mcp command builds its config from these same CE_ variables.