Install
Needs Node.js 22+ and Postgres with the vector, pg_trgm and unaccent extensions, which context-engine migrate creates when the database role can CREATE EXTENSION.
npm install @promptev/context-engine pg
Migrate the database first
The engine needs a Postgres with pgvector and its own tables. To try it locally, start one and migrate it, before any code runs:
docker run -d --name context-engine-pg -p 127.0.0.1:5432:5432 \
-e POSTGRES_USER=user -e POSTGRES_PASSWORD=pass -e POSTGRES_DB=mydb \
pgvector/pgvector:pg16
until docker exec context-engine-pg pg_isready -h localhost -U user -d mydb; do sleep 1; done
npx context-engine migrate --database-url postgresql://user:pass@localhost:5432/mydb --dim 1536
The second line waits until Postgres accepts connections: docker run -d returns before it does. The port is published on loopback only. If port 5432 is already taken, publish another one (-p 127.0.0.1:5433:5432) and use it in the URL (localhost:5433). migrate creates the extensions and the tables, and is safe to run again (see Create the tables). On a database that was never migrated, the first call throws DatabaseNotMigrated, and its message is the command to run:
DatabaseNotMigrated: This database has no Context Engine tables, or this role cannot see them. Run the migration once, then try again: npx context-engine migrate --database-url <your database URL> --dim 1536
DatabaseNotMigrated is an Error; the HTTP routers answer it with 503 and the MCP tools with one fixed sentence. The examples on this page use top-level await, so the project needs "type": "module" in its package.json.
Logging. The engine’s own per-document diagnostic lines (what an ingest embedded and reused, a document deleted mid-pass) are off by default. Run with NODE_DEBUG=context-engine to print them.
Peer dependencies by feature
Every peer is optional. Install only what the features you use need; a missing one throws ExtraMissingError naming the package.
| Feature | Packages | Needed for |
|---|
| Database driver | pg | Every deployment. |
| HTTP routers | hono, express, fastify, @fastify/multipart | The matching router factory. Fastify uploads need @fastify/multipart registered on your app. |
| MCP server | @modelcontextprotocol/server ^2.2.0, @modelcontextprotocol/node ^2.1.0 | createMcpApp and context-engine mcp. |
| MCP client | @modelcontextprotocol/client ^2.2.0 | Calling third-party MCP servers as mcp tools. Without it such a call fails with ToolExecutionFailed, whose message carries the install line, and testTool reports an error category. |
| Graph | graphology, graphology-communities-louvain, neo4j-driver | Graph mode. neo4j-driver is imported only when graph.neo4jUri is set. |
| Compute | danfojs-node, isolated-vm | compute(): dataframes and the sandbox. |
| Documents | pdfjs-dist, tesseract.js, sharp | Structure-preserving PDF text, OCR fallback, image handling for vision. |
@napi-rs/canvas is not a peer dependency, so npm will not suggest it; vision uses it to rasterise PDF pages, so install it yourself if you use vision.
npm install hono # createHonoRouter
npm install @modelcontextprotocol/client # mcp tools: calling third-party MCP servers
npm install @modelcontextprotocol/server @modelcontextprotocol/node # createMcpApp, context-engine mcp
npm install graphology graphology-communities-louvain # graph mode
npm install danfojs-node isolated-vm # compute()
Import
The package ships ESM and CommonJS builds with types for both.
import { ContextEngine, ContextEngineConfig, TRUSTED } from "@promptev/context-engine";
import { createHonoRouter } from "@promptev/context-engine/hono";
import { createMcpApp } from "@promptev/context-engine/mcp";
const { ContextEngine, ContextEngineConfig, TRUSTED } = require("@promptev/context-engine");
const { createExpressRouter } = require("@promptev/context-engine/express");
Configure
ContextEngineConfig is the one settings object. The constructor validates it: half a Neo4j config, a reranker naming a provider field, a provider the built-in client does not have, defaultMode: "graph" with graph off, or a hash redaction rule with no key all throw at construction; graph mode without graph.extractionLlm throws when the engine is built without a host model. Every field is on the configuration page.
import { ContextEngine, ContextEngineConfig } from "@promptev/context-engine";
const config = new ContextEngineConfig({
databaseUrl: "postgresql://user:pass@localhost:5432/mydb",
embedding: { provider: "openai", model: "text-embedding-3-small", apiKey: "sk-..." },
// optional: structured extraction, compute, map_reduce
llm: { provider: "openai", model: "gpt-4o-mini", apiKey: "sk-..." },
});
const engine = new ContextEngine(config); // cheap: no connection is opened yet
From the environment
ContextEngineConfig.fromEnv(overrides?) reads every CE_ variable. Nested fields use __ and names are camel-cased, so CE_EMBEDDING__API_KEY becomes embedding.apiKey. The values true and false become booleans and numeric strings become numbers. Overrides are deep-merged on top. It throws unless CE_DATABASE_URL and the embedding provider and model are set, by the environment or the overrides.
// CE_DATABASE_URL=postgresql://user:pass@localhost:5432/mydb
// CE_EMBEDDING__PROVIDER=openai
// CE_EMBEDDING__MODEL=text-embedding-3-small
// CE_EMBEDDING__API_KEY=sk-...
const config = ContextEngineConfig.fromEnv();
// explicit overrides are deep-merged over the environment
const tuned = ContextEngineConfig.fromEnv({ enableCodeExecution: true });
Create the tables
Run the migration once per database and again on each deploy. --dim must match your embedding width and is locked on the first run. It is safe to re-run after a failure; re-running is the repair.
npx context-engine migrate --database-url postgresql://user:pass@localhost:5432/mydb --dim 1536
npx context-engine migrate --database-url "$CE_DATABASE_URL" --dim 1536 --graph # also the graph tables
Or from code, for example in a deploy script:
import { runMigrate } from "@promptev/context-engine";
await runMigrate(process.env.CE_DATABASE_URL!, { dim: 1536, graph: false });
All CLI commands and flags are on the CLI page.
Ingest and search
import { TRUSTED } from "@promptev/context-engine";
const report = await engine.ingest({
text: "Annual leave accrues at two days per month.",
name: "leave-policy",
sourceId: "hr-handbook",
acl: ["group:hr"],
});
report.documents[0].status; // "completed", "skipped", "failed", ...
const result = await engine.search("how much annual leave do I get", {
sourceIds: ["hr-handbook"],
principals: ["group:hr"], // [] = anonymous caller, TRUSTED = skip ACL on purpose
topK: 10,
});
for (const hit of result.hits) console.log(hit.score, hit.documentName, hit.chunkText);
Never omit principals for a signed-out user. Omitted or null means trusted caller and returns the whole corpus. It is deprecated: it warns once per method, and a later release refuses it. Pass [] for an anonymous caller and TRUSTED when you mean it.
Close the engine
Construction opens nothing; the pool, embedder and graph store are built on first use. Release them with aclose(), or let await using do it at the end of a scope.
// 1. explicit
const engine = new ContextEngine(config);
try {
await engine.search("...", { principals: [] });
} finally {
await engine.aclose(); // safe to call more than once
}
// 2. scoped: ContextEngine implements Symbol.asyncDispose
{
await using scoped = new ContextEngine(config);
await scoped.stats();
} // aclose() runs here
Serve it
HTTP on Hono
createHonoRouter returns a Hono app with the document, search, stats, tool and approval routes. Mount it under any prefix. The routes are listed on the HTTP API page; Express and Fastify versions are in Router factories.
import { Hono } from "hono";
import { createHonoRouter } from "@promptev/context-engine/hono";
const app = new Hono();
const ce = createHonoRouter(engine, {
// runs before every route except the MCP OAuth callback; throw to refuse
auth: async (c) => {
if (!c.req.header("authorization")) throw new Error("unauthenticated");
},
// resolved per request, never read from the body: string[], [] or TRUSTED
principals: async (c) => ["user:alice", "group:hr"],
});
app.route("/ce", ce); // POST /ce/documents, POST /ce/search, GET /ce/stats, ...
MCP on node:http
createMcpApp is async and resolves to a (req, res) handler for streamable HTTP. It serves search_knowledge_base, search_tools and execute_tool. Both principals and scope are required.
import { createServer } from "node:http";
import { UNSCOPED } from "@promptev/context-engine";
import { createMcpApp } from "@promptev/context-engine/mcp";
// createMcpApp is async: await it before handing it to a server
const handler = await createMcpApp(engine, {
principals: () => ["user:alice"], // per call, never a tool argument
scope: ["hr-handbook"], // the ceiling of source ids; UNSCOPED = whole corpus on purpose
});
createServer((req, res) => {
void handler(req, res);
}).listen(8080, "127.0.0.1");
Package entry points
Each subpath is its own bundle so an app pulls only the peers it uses. Every entry point has ESM (import) and CommonJS (require) builds.
| Import path | Exports | Needs |
|---|
| @promptev/context-engine | The engine, config, errors, sentinels, helpers and types. ESM and CommonJS. | none beyond pg |
| @promptev/context-engine/hono | createHonoRouter, createHonoApp | hono |
| @promptev/context-engine/express | createExpressRouter | express |
| @promptev/context-engine/fastify | createFastifyPlugin | fastify, plus @fastify/multipart for uploads |
| @promptev/context-engine/mcp | createMcpApp | @modelcontextprotocol/server, @modelcontextprotocol/node |
| @promptev/context-engine/graph | Graph internals: buildGraphStore, detectCommunities, ENTITY_TYPES, normalizeEntityName, GraphStore and others | graphology, graphology-communities-louvain, neo4j-driver with a Neo4j |
| @promptev/context-engine/presidio | presidioDetector, presidioDetectors, DEFAULT_SCORE_THRESHOLD | a Presidio Analyzer at CE_PRESIDIO_URL, and curl on the host |
| bin context-engine | The CLI: migrate, mcp, check-acl-exposure, install-skill | see CLI |
ContextEngine constructor
Cheap and side-effect free: no connection is opened and no provider client is built until the first call that needs one. The engine exposes config and hooks as public fields.
new ContextEngine(config: ContextEngineConfig, opts?: {
onUsage?: ((e: UsageEvent) => void) | null;
onError?: ((e: unknown, ctx: Record<string, unknown>) => void) | null;
onProgress?: ((e: ProgressEvent) => void) | null;
onElicitation?: ElicitationHook | null;
codeRunner?: CodeRunner | null;
fetchFactory?: FetchFactory | null;
mcpLookup?: HostLookup | null;
model?: ModelFn | null;
// (texts, { kind }) => [vectors, tokens] | Promise<...>, or an object with that embed method
embedder?: HostEmbedFn | { embed: HostEmbedFn } | null;
// (query, docs) => number[] | Promise<number[]>
reranker?: RerankFn | null;
// { name: (query, { limit, scope }) => ids | Promise<ids> }
retrievalLegs?: Record<string, RetrievalLeg> | null;
// (ranked, weights, k) => ids | Promise<ids>
fusion?: FusionFn | null;
graphBackend?: GraphBackend | null;
})
| Option | Type | Default | What it does |
|---|
| config | ContextEngineConfig | required | The settings object. See Configuration. |
| onUsage | (e: UsageEvent) => void | null | Metering: one event per ingest, search and tool call. |
| onError | (e, ctx) => void | null | Errors the engine handled or degraded around, with context such as stage. |
| onProgress | (e: ProgressEvent) => void | null | Ingest stage boundaries for live progress. |
| onElicitation | (req: ElicitationRequest) => ElicitationResponse | null | A third-party MCP server’s question during an mcp tool call, sync or async. Unset, the client declares no elicitation. A server that asks anyway gets a protocol error instead of decline: on a 2026-07-28 connection that fails the call, and on a handshake connection the server decides what happens next. See Connecting MCP servers. |
| codeRunner | CodeRunner | null | null | (code, context, { timeout }) => SandboxResult. Runs compute code in your own isolation instead of the in-process sandbox. |
| fetchFactory | (purpose) => fetch | null | null | Your own long-lived fetch per purpose: llm, embeddings, reranker, tool_http, mcp_oauth. Return null to keep the engine’s. A host model, embedder or reranker never calls it. |
| mcpLookup | (hostname, port) => Promise<string[]> | system resolver | Your own name resolution for MCP connections. Answers are still address-checked. |
| model | ModelFn | null | null | Your own model behind every model call: (request: ModelRequest) => ModelReply | Promise<ModelReply>. request.purpose is one of structured_extraction, entity_extraction, community_summaries, vision_pages, vision_image, map_reduce, compute. With it, llm, visionLlm and graph.extractionLlm are optional. The engine never retries or times out a host call. See Bring your own model. |
| embedder | HostEmbedFn | { embed } | null | null | Your own embedding transport: (texts, { kind }) resolving to [vectors, tokens], or an object with that embed method. config.embedding stays required as the identity the database records (model, dim) and the batch limits; the dimension check runs on every reply. |
| reranker | RerankFn | null | null | Your own reranker: (query, docs) resolving to indices into docs, most relevant first. Replaces config.reranker on the same widened candidate window; out-of-range and duplicate indices are dropped and omitted ones appended. |
| retrievalLegs | Record<string, RetrievalLeg> | null | null | Extra retrieval legs by name: (query, { limit, scope }) returning chunk ids, sync or async, run beside the built-in legs. Their ids are re-filtered by the caller’s permissions before fusion. Weight fusion.weights[name], 1.0 when absent, 0 means never called. A name reusing fts, trgm, ann or graph throws. See Plug in your own retrieval. |
| fusion | FusionFn | null | null | Replaces reciprocal rank fusion: (ranked, weights, k) returning ids. Its answer is restricted to ids some leg produced; a fusion function that throws falls back to RRF. |
| graphBackend | GraphBackend | null | null | Supplies the raw graph: expandCandidates(seeds, queryEntities, { maxDepth, scope }) and chunkDegrees(ids, { scope }), value or promise. The engine applies the caller’s scope, then the caps (100 per entity, 500 in all), exactly as for the built-ins. Calls are bounded by search.pluginTimeoutS. |
const engine = new ContextEngine(config, {
onUsage: (e) => meter.record(e.kind, e.units, e.detail),
onError: (err, ctx) => log.error({ err, ...ctx }),
onProgress: (e) => ui.update(e.documentId, e.stage, e.state),
fetchFactory: (purpose) => (purpose === "llm" ? proxiedFetch : null),
// every model call the engine makes; request.purpose says which
model: async (request) => ({ text: "...", tokens: { input: 0, output: 0 } }),
});
// onToolCall is not a constructor option; hooks is a public field
engine.hooks.onToolCall = (event) => audit.push(event);
Ingest methods
Ingest is synchronous and idempotent per content hash: re-ingesting unchanged content is skipped, and a changed document re-embeds only the chunks whose text changed.
ingest
Ingest one document. Exactly one of file, content or text must be given; text skips extraction. Size and row caps (ingest.maxFileBytes, ingest.maxRows) are checked before any connection is opened. A failure while processing the document does not throw: it comes back as status: "failed" in the report. There is no principals option: ingest is a trusted, host-side call.
ingest(opts?: {
file?: unknown; // a path, a Buffer, a multer-style { buffer, originalname }, or a handle with read()
content?: Buffer | null;
filename?: string | null;
text?: string | null;
name?: string | null;
description?: string | null;
sourceId?: string | null;
externalId?: string | null;
metaData?: Record<string, unknown> | null;
acl?: string[] | null;
mode?: "hybrid" | "graph";
extractStructured?: boolean;
fieldHints?: Record<string, unknown>[] | null;
mimeType?: string | null;
}): Promise<IngestReport>
| Option | Type | Default | What it does |
|---|
| file / content / text | see signature | | The document. A string file is read from disk and its base name becomes the filename. |
| filename | string | null | null | Names content or text; the extension picks the extractor. |
| name, description | string | null | null | Caller-set attributes, never redacted on output. |
| sourceId, externalId | string | null | null | Your own identity for the document; re-ingesting the same identity updates it. |
| metaData | Record<string, unknown> | null | null | Free-form metadata stored on the document. |
| acl | string[] | null | null | Who may read it. null is unrestricted. |
| mode | "hybrid" | "graph" | config.defaultMode | "graph" requires graph.enabled. |
| extractStructured | boolean | false | Run structured field extraction; needs a host model or config.llm. |
| fieldHints | Record<string, unknown>[] | null | null | Fields to extract; needs extractStructured: true. |
| mimeType | string | null | null | What the document is, for example "text/plain" or "text/csv". It is believed over everything else. Without it the filename’s extension decides (the name stands in for it on a text ingest), and only a document with neither is judged by its content, where text is read as CSV only on strong evidence. The type picks the chunker, and a tabular one keeps the document out of graph mode. See how the file type is decided. |
Returns: IngestReport: { documents: DocumentReport[], totals: { files, failed, units, graphUnits } }. The error on each document report is masked by the configured output redaction policy.
Errors: Error unless exactly one of file, content or text is given; IngestTooLarge over a cap; Error for mode: "graph" with graph disabled, for extractStructured without a model, and for fieldHints without extractStructured.
const report = await engine.ingest({
file: "./handbook.pdf",
sourceId: "hr-handbook",
externalId: "handbook-2026",
acl: ["group:hr"],
});
const doc = report.documents[0];
if (doc.status === "failed") console.warn(doc.failureReason, doc.failureMessage);
resumeEmbeddings
Fill in the vectors an interrupted or failed run did not reach. Chunks are stored before they are embedded, so only rows still missing a vector are sent; nothing is re-extracted or re-chunked and the source file is not needed. Omit documentId to sweep resumable documents, oldest first (up to 100 per call). A document with nothing pending is completed and otherwise left alone, so this is safe to run from a cron job.
resumeEmbeddings(documentId?: string | null): Promise<IngestReport>
Returns: An IngestReport covering every document it touched.
await engine.resumeEmbeddings(documentId); // one document
await engine.resumeEmbeddings(); // every resumable document
resumeDocuments
Ask your model or embedder again for every document parked as waiting_model after it threw ModelDeferred. It re-runs the parked stage under the same request keys; a document that defers again parks again with rounds incremented. Omit documentIds to sweep the parked documents whose retry_at has passed (the database clock), oldest first, in pages of 100, each claimed with one compare-and-set, so a cron on several workers never asks you twice. Every call also fills the community summaries a source is still missing (that needs storage.poolMax of at least 2). See Batch mode.
resumeDocuments(opts?: {
documentIds?: string[] | null;
contentSource?: ContentSource | null;
}): Promise<ResumeReport>
| Option | Type | Default | What it does |
|---|
| documentIds | string[] | null | null | Resume these, due or not. One that is not parked is skipped without a word, so a resume is idempotent. |
| contentSource | ContentSource | null | null | For a document parked at stage extract: called with a ParkedDocument (documentId, sourceId, externalId, name, contentSha256), returns the original bytes as a Uint8Array, sync or async, or null. Without bytes the document stays parked with no_content_source; bytes whose sha256 differs leave it parked with content_mismatch. A stage enrich park needs no bytes. |
Returns: ResumeReport: { documents, skipped, superseded, communitiesResumed }. skipped is { documentId, reason } per document left parked on this call (no_content_source, content_mismatch, redaction_mismatch); superseded is one SupersededPark (documentId, requestKeys, supersededAt, reason: "superseded" or "failed") per set of keys nobody will ask for again, reported once.
const report = await engine.resumeDocuments({ contentSource: (doc) => blobs.get(doc.documentId) });
for (const park of report.superseded) await batch.drop(park.requestKeys); // nobody asks for these again
Search methods
Every read takes principals. Pass the caller’s list, [] for an anonymous caller, or TRUSTED when ACL filtering should be off on purpose.
search
Hybrid retrieval (full text, trigram and vector legs, fused) plus an optional graph leg. With mode omitted or null, the graph leg runs only when the deployment has a graph, it is reachable, and the query names an entity the caller can see. "graph" forces the leg; a query that names no entity still gets the hybrid answer. The graph leg adds to the hybrid legs, never replaces them.
search(query: string, opts?: {
sourceIds?: string[] | null;
documentIds?: string[] | null;
principals?: string[] | typeof TRUSTED;
topK?: number;
mode?: "hybrid" | "graph" | null;
compressToTokens?: number | null;
redaction?: RedactionPolicy | null;
secretKey?: string | null;
}): Promise<SearchResult>
| Option | Type | Default | What it does |
|---|
| sourceIds | string[] | null | null | Limit to these sources. |
| documentIds | string[] | null | null | Limit to these documents; intersects with sourceIds. |
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| topK | number | 10 | Hits to return (at least 1). |
| mode | "hybrid" | "graph" | null | null (decide per query) | Force or skip the graph leg. |
| compressToTokens | number | null | null | Compress the returned hits to fit this token budget. |
| redaction | RedactionPolicy | null | config.redaction | Output policy for this call only; overrides the configured one. |
| secretKey | string | null | redactionKey(config) | HMAC key for hash rules on this call. |
Returns: SearchResult: { hits: Hit[], usage }. usage carries kind, units, mode (asked for), modeUsed (what ran), graph ({ applied, reason, entities }, reason from GRAPH_REASONS), graphLeg, degraded, scopes, legs, legHits, candidates, returned, reranked, compressed, providerTokens, and redaction when a rule fired.
Errors: TypeError for a non-array principals; ExtraMissingError when graph mode is on but its peer packages are missing. With an explicit mode: "graph", a failure to resolve the query’s entities throws instead of falling back.
const { hits, usage } = await engine.search("what did the CFO say about Q3", {
principals: ["group:finance"],
mode: "graph",
});
usage.modeUsed; // "hybrid" when the query named no known entity
usage.graph.reason; // e.g. "no_entities_in_query"
documentsByName
Find documents by their name rather than their text; the knowledge tool’s search uses it when a query names a file. Metered like a search (one unit per scope) only when something matched; a name query that matched nothing is free and emits no usage event.
documentsByName(filename: string, opts?: {
sourceIds?: string[] | null;
documentIds?: string[] | null;
principals?: string[] | typeof TRUSTED;
limit?: number;
redaction?: RedactionPolicy | null;
secretKey?: string | null;
}): Promise<{ documents: DocumentByName[]; usage: Record<string, unknown> }>
| Option | Type | Default | What it does |
|---|
| sourceIds | string[] | null | null | Limit to these sources. |
| documentIds | string[] | null | null | Limit to these documents; intersects with sourceIds. |
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| limit | number | 10 | Maximum documents. |
| redaction | RedactionPolicy | null | config.redaction | Output policy for this call only; overrides the configured one. |
| secretKey | string | null | redactionKey(config) | HMAC key for hash rules on this call. |
Returns: documents: [{ document_id, name, score, snippet, source_id, chunk_idx }]; usage with kind: "search", matched_by: "filename", units, scopes, empty legs and legHits, and returned.
Document methods
A missing document and one the caller cannot see raise the same DocumentNotFoundError, so the API never confirms that a hidden document exists.
getDocument
One document with its full text. Output redaction masks text, documentType, structuredData and error; name, description and metaData are caller-set and left as stored.
getDocument(documentId: string, opts?: {
principals?: string[] | typeof TRUSTED;
redaction?: RedactionPolicy | null;
secretKey?: string | null;
}): Promise<Record<string, unknown>>
| Option | Type | Default | What it does |
|---|
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| redaction | RedactionPolicy | null | config.redaction | Output policy for this call only; overrides the configured one. |
| secretKey | string | null | redactionKey(config) | HMAC key for hash rules on this call. |
Returns: { id, sourceId, externalId, name, description, metaData, acl, mode, mimeType, lang, text, status, error, failureReason, chunks, documentType, structuredData, createdAt, updatedAt, startedAt, completedAt, unreadableReason, unreadablePages, embeddingProgress }
Errors: EngineActionError for an id that is not a UUID; DocumentNotFoundError when the document is missing or not visible (one message for both, so a caller cannot tell them apart).
const doc = await engine.getDocument(id, { principals: ["group:hr"] });
doc.status; // "embedding" is non-terminal: in flight or stopped part-way
doc.embeddingProgress; // { done, total, tokens } or null
getDocumentText
Just the text, untruncated, masked by the configured output policy. Token budgeting is the caller’s job.
getDocumentText(documentId: string, opts?: { principals?: string[] | typeof TRUSTED }): Promise<string>
| Option | Type | Default | What it does |
|---|
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
Returns: The document text.
Errors: Same as getDocument.
getDocuments
Several documents, each ACL-checked like getDocument. An id that is missing or not visible is left out of the result, never reported.
getDocuments(documentIds: string[], opts?: {
principals?: string[] | typeof TRUSTED;
redaction?: RedactionPolicy | null;
secretKey?: string | null;
}): Promise<Array<Record<string, unknown>>>
| Option | Type | Default | What it does |
|---|
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| redaction | RedactionPolicy | null | config.redaction | Output policy for this call only; overrides the configured one. |
| secretKey | string | null | redactionKey(config) | HMAC key for hash rules on this call. |
Returns: An array of the same objects getDocument returns.
getChunks
Read one document in order, a range of chunks at a time. At most 25 chunks per call; end is inclusive and clamped to that window.
getChunks(documentId: string, opts?: {
principals?: string[] | typeof TRUSTED;
start?: number | null;
end?: number | null;
redaction?: RedactionPolicy | null;
secretKey?: string | null;
}): Promise<Record<string, unknown>>
| Option | Type | Default | What it does |
|---|
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| start | number | null | 0 | First chunk position (a search hit’s chunk index works). |
| end | number | null | start + 24 | Last chunk position, inclusive. |
| redaction | RedactionPolicy | null | config.redaction | Output policy for this call only; overrides the configured one. |
| secretKey | string | null | redactionKey(config) | HMAC key for hash rules on this call. |
Returns: { document_id, total_chunks, start, end, chunks: [{ position, text, language }], has_more, next_start }
Errors: Same as getDocument.
let page = await engine.getChunks(id, { principals: TRUSTED });
while (page.has_more) {
page = await engine.getChunks(id, { principals: TRUSTED, start: page.next_start as number });
}
listDocuments
Page through visible documents, newest first, with keyset pagination. Output redaction masks documentType.
listDocuments(opts?: {
sourceId?: string | null;
sourceIds?: string[] | null;
documentIds?: string[] | null;
principals?: string[] | typeof TRUSTED;
cursor?: unknown;
limit?: number;
redaction?: RedactionPolicy | null;
secretKey?: string | null;
}): Promise<Record<string, unknown>>
| Option | Type | Default | What it does |
|---|
| sourceId | string | null | null | One source. |
| sourceIds | string[] | null | null | Limit to these sources. |
| documentIds | string[] | null | null | Limit to these documents; intersects with sourceIds. |
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| cursor | unknown | null | The previous page’s nextCursor. |
| limit | number | 50 | Page size, clamped to 1 to 200. |
| redaction | RedactionPolicy | null | config.redaction | Output policy for this call only; overrides the configured one. |
| secretKey | string | null | redactionKey(config) | HMAC key for hash rules on this call. |
Returns: { documents, count, hasMore, nextCursor? }. Each document: id, sourceId, externalId, name, description, mode, mimeType, lang, status, failureReason, documentType, createdAt, updatedAt, unreadableReason, unreadablePages, embeddingProgress, acl. nextCursor ({ time, id }) is present only when hasMore.
let page = await engine.listDocuments({ sourceId: "hr-handbook", principals: ["group:hr"] });
while (page.hasMore) {
page = await engine.listDocuments({
sourceId: "hr-handbook", principals: ["group:hr"], cursor: page.nextCursor,
});
}
updateDocument
Change caller-set attributes without re-sending content. PATCH semantics: only fields you pass change. acl: null makes the document unrestricted; omit the field (or pass UNSET) to leave it alone. metaData is merged into the stored object, not replaced. Passing acl always re-syncs the ACL onto the chunks.
updateDocument(documentId: string, opts?: {
acl?: string[] | null | typeof UNSET;
name?: string | null | typeof UNSET;
description?: string | null | typeof UNSET;
metaData?: Record<string, unknown> | null | typeof UNSET;
principals?: string[] | typeof TRUSTED;
}): Promise<string[]>
| Option | Type | Default | What it does |
|---|
| acl | string[] | null | UNSET | UNSET | Replaced; null unrestricts. |
| name, description | string | null | UNSET | UNSET | Replaced. |
| metaData | Record<string, unknown> | null | UNSET | UNSET | Merged. |
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
Returns: The fields that actually changed, from "acl", "name", "description", "meta_data". An empty array when nothing changed.
Errors: Same as getDocument; checked before anything is written.
const changed = await engine.updateDocument(id, {
acl: ["group:legal"],
metaData: { reviewed: true },
principals: ["group:hr"],
}); // ["acl", "meta_data"]
deleteDocument
Delete a document and all of its chunks. With graph mode on it also repairs the graph mirror the document leaves behind; a failure there is logged and sent to onError with stage: "delete_document_graph" and never fails the delete.
deleteDocument(documentId: string, opts?: { principals?: string[] | typeof TRUSTED }): Promise<void>
| Option | Type | Default | What it does |
|---|
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
Returns: Nothing.
Errors: Same as getDocument, checked before anything is deleted.
stats
Counts for the whole corpus or one source, as the caller sees it: TRUSTED counts everything, [] only unrestricted rows, a list the rows its acl overlaps. Omitted, it counts everything, as before.
stats(sourceId?: string | null, opts?: { principals?: Principals }): Promise<Record<string, unknown>>
Returns: { sourceId, documents, chunks, byStatus: { [status]: count }, embedding: { provider, model, dim } }
await engine.stats(); // whole corpus
await engine.stats("hr-handbook"); // one source
resyncGraph
Bring Neo4j back in line with the Postgres graph. Only a deployment with graph.neo4jUri set has anything to do here: Postgres is written first and is always complete, and a Neo4j write that still fails after its retries leaves that document out of Neo4j, reported as graphBackendStale on the document, through onError with stage graph_sync, and as graph_sync_pending in the document’s metadata. With no argument this replays every document recorded that way, each with its own entities and relationships, and retries every removal Neo4j still owes. full: true replays the whole corpus (every entity, every graph document, every relationship), then removes from Neo4j what Postgres no longer has; its cost follows the corpus. Run the full replay once after upgrading, and whenever the two stores must be made identical whatever happened before. Idempotent, calls no model, bills nothing. Needs graph mode. A maintenance call for the host: it takes no principals, and no router, tool or MCP action reaches it.
resyncGraph(opts?: {
documentId?: string | null;
sourceId?: string | null;
full?: boolean;
}): Promise<ResyncResult>
| Option | Type | Default | What it does |
|---|
| documentId | string | null | null | Replay that one document. A document that was not ingested in graph mode has nothing to replay. |
| sourceId | string | null | null | Replay every graph document of that source. |
| full | boolean | false | Replay the whole corpus and remove from Neo4j what Postgres no longer has. Takes no documentId or sourceId. |
Returns: { documents, synced, failed }, where failed lists document ids that keep their record. Three more keys appear only when they have something to say: removals: { done, failed }; with full, removed: { chunks, relationships, entities }; and error, a failure of a corpus-wide step of the full replay. Without a neo4jUri the result is all zeros.
Errors: InvalidInputError without graph mode, when both documentId and sourceId are given, or when full is combined with either. DocumentNotFoundError when documentId names no document.
await engine.resyncGraph(); // documents recorded as pending
await engine.resyncGraph({ sourceId: "hr-handbook" });
await engine.resyncGraph({ full: true }); // once after upgrading
rebuildCommunities
Detect, summarise and store one source’s communities again, reading and writing only that source. Use it for a source whose communities are missing (none built yet, or removed by a delete) and that gets no graph ingest to rebuild them. Needs graph mode. Summaries and their embeddings are written only with graph.communitySummaries on; off, it detects and stores the communities with no model call, bills 0 units, and drops the source’s existing summaries. Metered to onUsage as graph units with detail.operation === "rebuild_communities".
rebuildCommunities(sourceId: string | null): Promise<Record<string, unknown>>
Returns: { communities_detected, communities_summarized, primary_communities, units }
await engine.rebuildCommunities("hr-handbook");
enableBm25
Opt this database in to the experimental search.lexicalRank: "bm25". Installs four triggers on the chunk table (its lock is taken briefly, with a 2-second timeout and retries) and counts the chunks already stored without blocking ingest. Idempotent; running it again recounts. Until it has finished, a bm25 search ranks with ts_rank_cd, reports a LexicalRankFallback to onError and says so in usage.lexical. disableBm25() removes the triggers and statistics; compactBm25Stats() folds the statistics rows each write adds and never blocks a writer. Also npx context-engine bm25-enable.
enableBm25(): Promise<Bm25EnableReport>
Returns: { createdTriggers, sources: [{ sourceId, chunks }] }
await engine.enableBm25();
Corpus and structured data
The methods behind the knowledge tool’s discover and query_meta actions. Each is scoped and ACL-filtered like a search.
spreadsheetSchema
Sheet names, column headers and row counts for the named spreadsheets, so a model can write one correct compute call. ACL and scope are applied again; a document that is not visible, not in scope or not tabular is simply absent. An empty documentIds returns an empty array. Output redaction masks sheet names and headers.
spreadsheetSchema(opts: {
documentIds: string[];
sourceIds?: string[] | null;
principals?: string[] | typeof TRUSTED;
redaction?: RedactionPolicy | null;
secretKey?: string | null;
}): Promise<SpreadsheetDescription[]>
| Option | Type | Default | What it does |
|---|
| documentIds | string[] | required | The documents to describe. |
| sourceIds | string[] | null | null | Limit to these sources. |
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| redaction | RedactionPolicy | null | config.redaction | Output policy for this call only; overrides the configured one. |
| secretKey | string | null | redactionKey(config) | HMAC key for hash rules on this call. |
Returns: [{ document_id, name, source_id, sheets, schema_unavailable? }]. sheets is null (with a reason) for a document past the read budget.
documentStructure
What is inside each named document whatever its type: sheets and columns for a workbook, sections and the last page for a document with headings, top-level keys for JSON, and a chunk count for everything.
documentStructure(opts: {
documentIds: string[];
sourceIds?: string[] | null;
principals?: string[] | typeof TRUSTED;
bounded?: boolean;
redaction?: RedactionPolicy | null;
secretKey?: string | null;
}): Promise<Record<string, DocumentStructure>>
| Option | Type | Default | What it does |
|---|
| documentIds | string[] | required | The documents to describe. |
| sourceIds | string[] | null | null | Limit to these sources. |
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| bounded | boolean | true | Cap every list at 40 items and report the rest; false returns everything. |
| redaction | RedactionPolicy | null | config.redaction | Output policy for this call only; overrides the configured one. |
| secretKey | string | null | redactionKey(config) | HMAC key for hash rules on this call. |
Returns: An object keyed by document id.
tabularScope
Whether everything in scope is a spreadsheet, and what the first one looks like. One grouped query over the document table.
tabularScope(opts?: {
sourceIds?: string[] | null;
documentIds?: string[] | null;
principals?: string[] | typeof TRUSTED;
redaction?: RedactionPolicy | null;
secretKey?: string | null;
}): Promise<TabularScope>
| Option | Type | Default | What it does |
|---|
| sourceIds | string[] | null | null | Limit to these sources. |
| documentIds | string[] | null | null | Limit to these documents; intersects with sourceIds. |
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| redaction | RedactionPolicy | null | config.redaction | Output policy for this call only; overrides the configured one. |
| secretKey | string | null | redactionKey(config) | HMAC key for hash rules on this call. |
Returns: { documents, tabular, all_tabular, sheet, frame_key }
documentTypes
A census of the visible corpus. kind comes from the mime type and is always known; type is the LLM-written document type and exists only where structured extraction ran.
documentTypes(opts?: {
sourceIds?: string[] | null;
documentIds?: string[] | null;
principals?: string[] | typeof TRUSTED;
redaction?: RedactionPolicy | null;
secretKey?: string | null;
}): Promise<DocumentTypeCount[]>
| Option | Type | Default | What it does |
|---|
| sourceIds | string[] | null | null | Limit to these sources. |
| documentIds | string[] | null | null | Limit to these documents; intersects with sourceIds. |
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| redaction | RedactionPolicy | null | config.redaction | Output policy for this call only; overrides the configured one. |
| secretKey | string | null | redactionKey(config) | HMAC key for hash rules on this call. |
Returns: [{ kind: "spreadsheet" | "text", type, documents, with_fields }]
fieldSummary
Extracted structured field names grouped by the same (kind, type) pair documentTypes uses, counted over the documents the caller can see.
fieldSummary(opts?: {
sourceIds?: string[] | null;
documentIds?: string[] | null;
principals?: string[] | typeof TRUSTED;
redaction?: RedactionPolicy | null;
secretKey?: string | null;
}): Promise<FieldGroup[]>
| Option | Type | Default | What it does |
|---|
| sourceIds | string[] | null | null | Limit to these sources. |
| documentIds | string[] | null | null | Limit to these documents; intersects with sourceIds. |
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| redaction | RedactionPolicy | null | config.redaction | Output policy for this call only; overrides the configured one. |
| secretKey | string | null | redactionKey(config) | HMAC key for hash rules on this call. |
Returns: [{ kind, type, fields: [{ field, type, documents }], more_fields? }]. more_fields lists the names past the per-group cap and is absent when nothing was left out.
queryStructured
Filter documents by the structured fields extraction stored for them. The question is resolved to known field names first.
queryStructured(question: string, opts?: {
sourceIds?: string[] | null;
principals?: string[] | typeof TRUSTED;
docType?: string | null;
limit?: number;
redaction?: RedactionPolicy | null;
secretKey?: string | null;
}): Promise<Record<string, unknown>>
| Option | Type | Default | What it does |
|---|
| sourceIds | string[] | null | null | Limit to these sources. |
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| docType | string | null | null | Only this document type. |
| limit | number | 20 | Maximum documents, clamped to 1 to 200. |
| redaction | RedactionPolicy | null | config.redaction | Output policy for this call only; overrides the configured one. |
| secretKey | string | null | redactionKey(config) | HMAC key for hash rules on this call. |
Returns: { question, resolvedFields, documents, count }
Errors: EngineActionError for an empty question.
Compute and map reduce
The two expensive reads: code over every row, and one model call per document.
compute
Answer a question over every row of the in-scope spreadsheets (CSV, TSV, XLSX): an LLM writes code, which runs over dataframes in a sandbox, with one retry on failure. Off by default: set enableCodeExecution: true only where execution is isolated. Pass a codeRunner to the constructor to run the code in your own isolation boundary. Needs the danfojs-node and isolated-vm peers.
compute(instruction: string, opts?: {
sourceIds?: string[] | null;
documentIds?: string[] | null;
principals?: string[] | typeof TRUSTED;
modelCfg?: LLMConfig | null;
timeout?: number;
redaction?: RedactionPolicy | null;
secretKey?: string | null;
model?: ModelFn | null;
}): Promise<Record<string, unknown>>
| Option | Type | Default | What it does |
|---|
| sourceIds | string[] | null | null | Limit to these sources. |
| documentIds | string[] | null | null | Limit to these documents; intersects with sourceIds. |
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| modelCfg | LLMConfig | null | null | A built-in provider config for the model that writes the code. Most specific first: the per-call model, then this, then the engine’s model, then config.llm. |
| model | ModelFn | null | null | A host function that writes the code for this call only, beating modelCfg and the engine’s model. |
| timeout | number | 30 | Seconds per execution, clamped to 1 to 300. |
| redaction | RedactionPolicy | null | config.redaction | Output policy for this call only; overrides the configured one. |
| secretKey | string | null | redactionKey(config) | HMAC key for hash rules on this call. |
Returns: { success, result, code, stdout, error, executionTime, documentsUsed, providerTokens: { llm_input, llm_output }, attempts }
Errors: EngineActionError when code execution is disabled, when no tabular document is in scope, or when more than 50 are; Error with no LLM configured; ExtraMissingError without danfojs-node.
const out = await engine.compute("total revenue by region for 2025", {
sourceIds: ["finance"],
principals: ["group:finance"],
});
if (out.success) console.log(out.result);
mapReduce
Ask the same question of every document in scope, one LLM call per document, newest first. A document that fails is reported in the results, not thrown.
mapReduce(instruction: string, opts?: {
principals?: string[] | typeof TRUSTED;
sourceIds?: string[] | null;
documentIds?: string[] | null;
limit?: number | null;
maxConcurrency?: number | null;
redaction?: RedactionPolicy | null;
secretKey?: string | null;
}): Promise<Record<string, unknown>>
| Option | Type | Default | What it does |
|---|
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| sourceIds | string[] | null | null | Limit to these sources. |
| documentIds | string[] | null | null | Limit to these documents; intersects with sourceIds. |
| limit | number | null | 25 | Documents to read, clamped to 1 to 200. |
| maxConcurrency | number | null | 5 | Calls in flight, clamped to 1 to 20. |
| redaction | RedactionPolicy | null | config.redaction | Output policy for this call only; overrides the configured one. |
| secretKey | string | null | redactionKey(config) | HMAC key for hash rules on this call. |
Returns: { results, processed, failed, considered }. Each result carries document_id and document_name with data, or an error.
Errors: EngineActionError when no LLM is configured.
mapReduceTargets
The document ids mapReduce would read, newest first, without calling a model.
mapReduceTargets(opts?: {
principals?: string[] | typeof TRUSTED;
sourceIds?: string[] | null;
documentIds?: string[] | null;
limit?: number | null;
}): Promise<string[]>
| Option | Type | Default | What it does |
|---|
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| sourceIds | string[] | null | null | Limit to these sources. |
| documentIds | string[] | null | null | Limit to these documents; intersects with sourceIds. |
| limit | number | null | 25 | Clamped to 1 to 200. |
Returns: An array of document ids.
Knowledge tool
One tool with actions, for an agent to use the corpus.
searchKnowledgeBase
The one knowledge tool as a library call: the same function the MCP server serves, for a host running its own agent loop. Arguments use the tool’s snake_case names. scope is required: pass the source ids this call may reach, or UNSCOPED to mean the whole corpus on purpose; ids the model asks for outside the ceiling are dropped. The actions and their answers are documented on the knowledge tool page.
searchKnowledgeBase(args: {
action: string; // one of KNOWLEDGE_ACTIONS
principals?: string[] | typeof TRUSTED;
scope: ScopeInput; // required ceiling: source ids, a Scope, or UNSCOPED
query?: string | null;
document_id?: string | null;
source_ids?: string[] | null;
document_ids?: string[] | null;
entity?: string | null;
depth?: number | null;
category?: string | null;
label?: string | null;
entity_type?: string | null;
top_k?: number | null;
mode?: string | null;
limit?: number | null;
cursor?: unknown;
start?: number | null;
end?: number | null;
max_chars?: number | null;
redaction?: RedactionPolicy | null;
secret_key?: string | null;
compute?: KnowledgeComputeFn | null;
map_reduce?: ((instruction: string, opts: Record<string, unknown>) => Promise<Record<string, unknown>>) | null;
}): Promise<Record<string, unknown>>
Returns: The action’s answer. An action the deployment cannot serve answers { success: false, error }.
Errors: Error when scope is null or omitted.
import { knowledgeToolDefinition } from "@promptev/context-engine";
const tool = knowledgeToolDefinition(); // { name, description, input_schema, annotations }
// inside your agent loop, when the model calls the tool:
const answer = await engine.searchKnowledgeBase({
...modelArgs, // action, query, document_id, ...
principals: ["user:alice"], // from your session, never from the model
scope: ["hr-handbook"],
});
Tool methods
Governed tools: stored http, db and mcp tools with ACL, approvals and an audit row per call, plus in-memory function tools.
executeTool
Run a visible tool and write an audit row. A tool that requires approval (or whose approvalPolicy.condition matches the arguments) does not run: it returns an approval_required record until the approval is resolved, then the same call runs once. Argument keys starting with an underscore are stripped. Every call whose tool was started leaves one audit row: if the full row cannot be written a last-resort row is (input_args set to { "_unstorable": true }, no result, output_truncated true, an error note naming what failed), and it depends on no other row. A tool or an approval deleted while the call ran costs the row only its reference: the row keeps its real arguments and result, with tool_id and approval_id NULL and a note in error_message, so filter audit rows on success, not on error_message. A tool that throws anything is a failed call with a failed row. JavaScript cannot cancel a promise: a caller that races executeTool against a timeout only stops waiting, and the call runs on to its audit row, so the Python package’s tools.audit_cancel_wait_s has no counterpart here. What cannot be recorded is a process that dies between the tool running and the insert. On the route handlers, over MCP and in the audit row a result keeps its JSON form: a Date is ISO 8601 text, and a value with a toJSON method is what that returns. Only a value JSON cannot carry is changed: a lone surrogate becomes U+FFFD, a non-finite number the text "NaN", "Infinity" or "-Infinity", U+0000 is removed, a bigint becomes its digits as text, and a property whose value is undefined is sent as null. A generator, an iterator or an async iterator has no JSON form and is sent and stored as {}: return an array. Called in process, the tool’s own value is returned unchanged. An approval matches its arguments exactly, by canonical JSON form: key order does not matter, and 1, true and "1" are three different arguments, while two spellings of one number are the same argument (1e16 and 10000000000000000, 0 and -0).
executeTool(callName: string, args: Record<string, unknown> | null, opts?: {
sourceId?: string | null;
principals?: string[] | typeof TRUSTED;
actor?: Record<string, unknown> | null;
source?: string;
approvalScope?: string | null;
resultMaxChars?: number | null;
resultMaxRows?: number | null;
responseMode?: "json" | "tsv" | null;
redaction?: RedactionPolicy | null;
secretKey?: string | null;
toolId?: string | null;
overrides?: { headers?: Record<string, string>; query?: Record<string, unknown>; body?: Record<string, unknown> } | null;
}): Promise<Record<string, unknown>>
| Option | Type | Default | What it does |
|---|
| sourceId | string | null | null | Which source’s tools to look in. |
| principals | string[] | [] | TRUSTED | omitted (deprecated) | The caller’s principals for ACL filtering. [] is an anonymous caller that sees only unrestricted rows; TRUSTED disables filtering on purpose. Omitting it or passing null also means trusted, emits a DeprecationWarning (CE_PRINCIPALS_NULL) once per method, and a later release refuses it. A non-array value throws TypeError. |
| actor | { type?, id? } | null | null | Recorded on the audit row. |
| source | string | "api" | Recorded on the audit row. |
| approvalScope | string | null | null (deprecated) | Opaque claim scope for approvals, such as a run id; resolve it server side. Omitted is the legacy unscoped path and warns. |
| resultMaxChars | number | null | 8000 | Budget for the returned result; null is no limit. |
| resultMaxRows | number | null | 100 | Rows per table in the result; null is no limit. |
| responseMode | "json" | "tsv" | null | tool config, else "json" | TSV turns every array of objects into a TSV string. |
| redaction | RedactionPolicy | null | config.redaction | Output policy for this call only; overrides the configured one. |
| secretKey | string | null | redactionKey(config) | HMAC key for hash rules on this call. |
| toolId | string | null | null | Picks one tool when several visible tools share the call name. |
| overrides | { headers?, query?, body? } | null | null | This call’s values over what the tool was saved with. An override wins over a value the model would fill. Host-only, never stored or audited. See One call’s values. |
Returns: { result, usage: { units: 1, kind: "tool", tool_name, tool_id, truncated } }, or { approval_required: { approval_id, tool_name, tool_id, args, reason, expires_at } }.
Errors: EngineActionError tool not found (unknown, invisible or wrong id all read the same) and tool execution failed; AmbiguousToolError when the name matches more than one visible tool and no toolId is given. InvalidInputError before the tool runs, and before any approval is opened, when an argument cannot be stored: broken Unicode (a lone surrogate), a non-finite number, nesting deeper than 64 levels, or a value that is not a JSON value (a Map, a Date, a function, any class instance); the message names the argument’s path and never its value. A NUL character (U+0000) in an argument is removed, not refused. ToolAuditFailed when the tool ran and no audit row could be written at all, with tools.auditOnFailure at "fail": it carries the result, so do not send the call again.
import { resolveApproval } from "@promptev/context-engine";
const out = await engine.executeTool("http_get_weather", { city: "Lahore" }, {
principals: ["user:a"],
approvalScope: "run-42",
});
if ("approval_required" in out) {
const { approval_id } = out.approval_required as { approval_id: string };
await resolveApproval(engine, approval_id, "approved", "boss");
// the same call with the same approvalScope runs, exactly once
}
Redaction views and closing
Per-tenant views and cleanup.
withRedaction
A view of the engine for one tenant: every method on it applies policy (its ingest and output rules) and hashes with secretKey, while sharing the pool, embedder, graph store, hooks, function tools and host seams. A per-call redaction option still overrides it. The view snapshots the config when built, does not re-redact stored content, and does not scope function tools. Closing a view is a no-op.
withRedaction(policy: RedactionPolicy, secretKey?: string | null): ContextEngine
Returns: A new ContextEngine view.
Errors: TypeError when policy is not a RedactionPolicy (pass new RedactionPolicy() to turn redaction off); Error for an empty secretKey, or a hash rule with no key at all. Falling back to config.secretKey for a hash rule logs one warning per engine.
import { RedactionPolicy } from "@promptev/context-engine";
const tenant = engine.withRedaction(
new RedactionPolicy({ rules: [{ name: "email", detector: "email", action: "hash" }] }),
tenantSecret,
);
await tenant.search("invoice from acme", { principals: ["tenant:acme"] });
aclose
Release what the engine opened: the embedder, the graph store and the Postgres pool. Construction is lazy, so only what was built is torn down. Safe to call more than once; a later call rebuilds lazily. On a withRedaction view it does nothing.
aclose(): Promise<void>
[Symbol.asyncDispose](): Promise<void>
Returns: Nothing.
Hooks
Pass onUsage, onError, onProgress and onElicitation to the constructor; set onToolCall on engine.hooks, since the constructor does not take it. A hook that throws is caught and logged, never raised into the engine. onToolCall receives the audit row of each tool call; embedding sends a progress event per completed batch with detail { done, total, tokens }.
interface Hooks {
onUsage?: ((event: UsageEvent) => void) | null;
onError?: ((exc: unknown, ctx: Record<string, unknown>) => void) | null;
onToolCall?: ((event: Record<string, unknown>) => void) | null;
onProgress?: ((event: ProgressEvent) => void) | null;
// A third-party MCP server's question during an mcp tool call; unset declares nothing.
onElicitation?: ((req: ElicitationRequest) => ElicitationResponse | Promise<ElicitationResponse>) | null;
}
interface ProgressEvent {
sourceId: string | null;
externalId: string | null;
documentId: string | null; // null until the row is claimed
name: string;
stage: "extract" | "redact" | "chunk" | "embed" | "structured" | "graph";
state: "started" | "done" | "progress"; // ignore states you do not recognise
detail: Record<string, unknown>;
}
Sentinels
Three symbols, created with Symbol.for so they compare equal across the package’s separate bundles.
TRUSTED
A principals value meaning trusted caller, ACL filtering off. Say it explicitly; null and omitted are the deprecated spellings.
UNSET
“Argument omitted”, distinct from null, for updateDocument fields.
UNSCOPED
A knowledge-tool scope meaning no ceiling. A null scope throws instead of meaning the whole corpus.
resolvePrincipals
resolvePrincipals(value, method) is exported: it turns a public value into the internal form and throws TypeError for a non-array.
import { TRUSTED, UNSET, UNSCOPED } from "@promptev/context-engine";
await engine.search("q", { principals: TRUSTED }); // ACL off, on purpose
await engine.search("q", { principals: [] }); // anonymous: unrestricted rows only
await engine.updateDocument(id, { acl: null, principals: TRUSTED }); // unrestricts
await engine.updateDocument(id, { acl: UNSET, principals: TRUSTED }); // leaves acl alone
await createMcpApp(engine, { principals: () => [], scope: UNSCOPED }); // whole corpus, on purpose
Errors
All are exported from the package root and set name, so both instanceof and err.name work. Configuration and programming errors stay as TypeError, RangeError or Error. The last column says whether the Python package exports a class of the same name from its root (Submodule: it exists in Python, but only in a submodule).
| Error | When | Python root export |
|---|
| EngineActionError | A data-dependent failure: invalid id, tool not found, tool execution failed, compute disabled, no LLM for map reduce. | Yes |
| DatabaseNotMigrated | An Error: the first call on a database with none of the engine’s tables, or whose role cannot see them. Its message is the migrate command to run. It replaces the driver’s undefined-table error (a pg error with code 42P01). Routers answer 503 with one fixed sentence, and so do the MCP tools. | Yes |
| ToolExecutionFailed | Subclass of EngineActionError: a tool that exists ran and failed. A tool that throws a value that is not an Error (throw undefined, throw null, Promise.reject()) fails with the text the tool threw a non-error value. Routers answer 502. | Yes |
| ToolAuditFailed | Subclass of ToolExecutionFailed: a tool call could not be recorded in the audit log at all, neither its full row nor the last-resort row (the database is down). The tool has already run. Thrown only with tools.auditOnFailure at "fail", the default. Carries toolRan (always true), result (shaped and redacted as the caller would have received it), truncated and toolError. Do not retry it blindly: that runs the tool a second time. Routers answer 502 with tool_ran: true and the result in the body, and the MCP surface answers an error whose text says the tool ran and ends with the result. The call is metered (onUsage fires), because the tool ran. | Yes |
| InvalidInputError | The caller’s own input, refused on purpose: an empty query, an unknown mode, a bad cursor, a missing scope, a tool argument that cannot be stored, a NUL character in a field the caller supplies. In the routers a search the engine refuses (an empty query, an unknown mode) is a 400. Its toolRan is false: when executeTool throws it the tool did not run and the call is safe to send again once corrected. Routers answer 400. On the MCP surface it, EngineActionError and DocumentNotFoundError are the only errors whose text reaches the client. | Yes |
| AmbiguousToolError | Subclass of EngineActionError: a call name shared by several visible tools and no toolId. Routers answer 409. | Yes |
| DocumentNotFoundError | Missing or not visible, with one message: document not found: <id>. Also re-exported from the engine module. | No |
| IngestTooLarge | Over ingest.maxFileBytes or ingest.maxRows. Carries knob, limit and actual. Routers answer 413. | Yes |
| ExtractionFailed | The file could not be read; the original error is its cause. | Yes |
| ExtraMissingError | An optional peer package is missing. The message names the package and the npm install to run. | No |
| GraphLegUnavailable | The graph ranked list was not supplied. | Yes |
| ApprovalNotPending | Resolving an approval that is not pending (or not visible: message not_found). | Yes |
| ApprovalExpired | Resolving an approval past its expiry. | Yes |
| CodeExecutionError | Generated code failed to run. | Submodule |
| CodeExecutionTimeout | Generated code ran past its timeout. | Submodule |
| SchemaError | A JSON Schema outside the portable subset. | Submodule |
| SchemaValidationError | A value broke its schema; carries path and detail. | Submodule |
| StructuredCallError | A model reply could not be made to satisfy its schema; carries mode, attempts, detail, tokens, and a non-enumerable raw. | Submodule |
| StructuredRefusal | Subclass of StructuredCallError: the provider signalled a refusal. | Submodule |
| ModelDeferred | Thrown by YOUR model or embedder, not by the engine: the request was accepted and its answer comes later. new ModelDeferred({ retryAfterMs }) takes milliseconds; a negative or non-finite value throws RangeError. During ingest the document parks as waiting_model; at query time it is refused for map_reduce and compute, and a query embedding deferral is treated as an embedder outage. | Yes |
| OpenAICompatError | The built-in OpenAI-compatible client got a non-retryable status, or ran out of retries. Carries status, body (the response text) and headers (the response headers only, so no request header and no API key is on the error). A connection failure or timeout is the rejected fetch or the per-attempt TimeoutError instead. | Yes |
| OpenAICompatReplyError | A successful reply that does not carry what was asked for: a chat reply with no choice, or an embeddings reply that does not hold exactly one vector per input with each index used once. Thrown in place of an empty answer. | Yes |
| EmbeddingBatchRejected, EmbedBatchFailed | Embedding provider failures during ingest; they surface as failed documents in the report. | Yes |
| InputTooLarge | A single chunk is over embedding.maxInputTokens. A chunk over it is never split: inside an ingest the document fails with failureReason: "input_too_large", and embedChunks throws this error when you call it yourself. | Yes |
| EmbeddingDimensionMismatch | An Error: an embedder, yours or the built-in client’s, returned vectors of a different width than embedding.dim or than this database’s vector columns. The document fails with failureReason: "embedding_dimension_mismatch". | Yes |
Router factories
Three adapters over one set of handlers, with the same 21 routes and the same options. Each throws TypeError without auth and principals, and ExtraMissingError when its framework is not installed. Handler errors answer { detail } with the right status. Routes, bodies and status codes are on the HTTP API page.
createHonoRouter / createHonoApp
From /hono. Returns a Hono app; createHonoApp is the same function. Parses multipart uploads itself.
createExpressRouter
From /express. Returns an Express Router. You own body parsing: express.json() and a multer middleware that puts the upload on req.file or req.files.file.
createFastifyPlugin
From /fastify. Returns a plugin named context-engine that runs auth as a preHandler hook. Register @fastify/multipart for uploads.
Upload size
Every adapter checks Content-Length against ingest.maxFileBytes and answers 413. Wire uploadLimitBytes(engine) into your own body limits to stop the bytes earlier.
| Option | Type | Default | What it does |
|---|
| auth | (ctx) => unknown | Promise<unknown> | required | Runs before every route except /mcp/oauth/callback. Throw to refuse the request. |
| principals | (ctx) => string[] | TRUSTED | required | Resolved per request, never from the body. Returning null or a non-array is a 500: return [] for an anonymous caller. |
| approvalScope | (ctx) => unknown | unset (deprecated) | Server-side approval claim scope for POST /tools/execute. Omitted is the unscoped legacy path. |
| redirectBaseUrl | string | unset | Public base URL of the mount; /mcp/oauth/callback is appended for MCP OAuth. POST /mcp/oauth/start answers 500 without it. |
ctx is the Hono context, the Express request or the Fastify request.
On another framework. createToolsHandlers(engine, { redirectBaseUrl, pendingStore, configTemplate }), from the package root, returns the framework-neutral tool-plane handlers the three adapters mount (ToolsHandlers). Each handler takes the caller’s principals as an argument, and execution takes the approvalScope beside them, so you resolve auth, principals and the approval scope in your own server exactly as the adapters do. configTemplate defaults to false; pendingStore defaults to an in-memory store, so pass a PostgresPendingStore for several workers. For documents and search, call engine.ingest() / engine.search() from your own handlers; the engine needs no web framework.
import express from "express";
import multer from "multer";
import { uploadLimitBytes } from "@promptev/context-engine";
import { createExpressRouter } from "@promptev/context-engine/express";
const app = express();
app.use(express.json({ limit: uploadLimitBytes(engine) ?? undefined }));
app.use(multer({ limits: { fileSize: uploadLimitBytes(engine) ?? undefined } }).single("file"));
app.use("/ce", createExpressRouter(engine, { auth: requireUser, principals: principalsOf }));
import Fastify from "fastify";
import multipart from "@fastify/multipart";
import { createFastifyPlugin } from "@promptev/context-engine/fastify";
const app = Fastify();
await app.register(multipart);
await app.register(createFastifyPlugin(engine, { auth: requireUser, principals: principalsOf }), {
prefix: "/ce",
});
MCP app
createMcpApp(engine, opts) from /mcp resolves to an async (req, res) handler for streamable HTTP, with the MCP server on its mcp property. It registers search_knowledge_base (see the knowledge tool), search_tools and execute_tool. It throws TypeError without principals or scope, and ExtraMissingError without @modelcontextprotocol/server and @modelcontextprotocol/node. It builds a fresh server per request (the MCP stateless model), answers the 2026-07-28 protocol version and the older initialize handshake from the same tools, and issues no session id. allowedHosts turns on DNS-rebinding protection for exactly those Host values; an empty list or a * wildcard is refused. A crash reaches the client as Error executing tool <name> and its real text goes to console.error. Every option may be a function resolved fresh on each call, so one mount can serve many tenants.
| Option | Type | Default | What it does |
|---|
| principals | () => unknown | required | Zero-argument, sync or async, resolved per call. Never a tool argument. |
| scope | ScopeInput | () => ScopeInput | required | The ceiling of source ids (or a Scope) the tool may reach. UNSCOPED means the whole corpus on purpose; null throws TypeError. |
| redaction | RedactionPolicy | () => RedactionPolicy | config.redaction | Per-call output policy. |
| secretKey | string | null | () => string | null | redactionKey(config) | Per-call hash key. |
| compute | KnowledgeComputeFn | null | null | Replaces the built-in compute action, and makes it available whatever enableCodeExecution says. Its result is masked on the engine side. |
| mapReduce | (instruction, opts) => Promise<object> | null | null | Replaces the built-in map_reduce action. |
| approvalScope | () => unknown | null | Approval claim scope for execute_tool. |
The tools are listed with JSON Schema 2020-12 input schemas ($schema is https://json-schema.org/draft/2020-12/schema), and an integer argument carries the safe-integer bounds as minimum and maximum.
Calling third-party MCP servers
The other direction, an mcp tool that calls someone else’s MCP server, runs on the official MCP SDK client, @modelcontextprotocol/client (an optional peer, ^2.2.0), over Streamable HTTP only. It speaks protocol versions 2026-07-28, 2025-11-25, 2025-06-18, 2025-03-26, 2024-11-05 and 2024-10-07. The global settings live under tools.mcp in the config (protocol, redirects, elicitationUrl, elicitationTimeoutS, redactLogs), and a tool’s own config may override all but redactLogs for its connection. Settings, redirects, elicitation, guarantees and error texts are in Connecting MCP servers.
Other exports
Functions and classes on the package root beyond the engine itself.
| Export | What it does |
|---|
| runMigrate(databaseUrl, { dim?, graph? }) | Create or upgrade the schema from code; the same work as context-engine migrate. dim defaults to 1536 and is locked on first run (a different dim on an existing database throws); graph defaults to false. |
| InProcessRunner, CeleryRunner | TaskRunner implementations: submit(factory) returns a task id and status(id) returns { state, error, result }. InProcessRunner works in one process; CeleryRunner is a documented seam whose methods throw NotImplementedError. |
| loadSchema(name), validate(value, schema, { lint? }), strictPortable(schema) | The shipped response schemas (community_summaries, entity_extraction, structured_envelope, vision_pages), the validator and the portable-subset linter. Throw SchemaError or SchemaValidationError. |
| structuredCall(cfg, { system, user, schema, schemaName, enforce?, ... }) | One LLM call whose reply must satisfy a JSON Schema the provider enforces where it can. Returns { value, mode, tokens, attempts, refusal }; mode is native, tool or prompt. A failure throws StructuredCallError, whose raw (the full final reply) is non-enumerable on purpose, so util.inspect and JSON.stringify skip it; read err.raw deliberately. structuredRepairCount() and resetStructuredRepairCount() count replies that needed the repair path. |
| checkIntakeSize(nbytes, limit, what?) | Throw IngestTooLarge when nbytes is over limit; null lifts it. checkRowLimit is the row twin. DEFAULT_MAX_FILE_BYTES is 256 MiB. |
| countCsvRows(content) | Rows in a CSV or TSV buffer, counting a newline inside a quoted field as part of its row. Also countXlsxRows, countTabularRows and checkTabularRows. |
| uploadLimitBytes(engine), checkContentLength(engine, header) | The byte cap the handlers enforce, for your own upload middleware (multer limits.fileSize, Fastify bodyLimit), and the Content-Length check itself, which throws a 413 handler error. |
| resolveScope(value), narrowToCeiling(requested, ceiling), KNOWLEDGE_ACTIONS | Scope helpers of the knowledge tool: normalise a host ceiling (null throws), and intersect a request with it (never widens). KNOWLEDGE_ACTIONS is the action list; knowledgeToolDefinition() and callKnowledgeTool are exported too. |
| redactHits(hits, policy, { principals, secretKey, hooks? }) | Apply an output policy to search hits yourself. Returns [hits, note]; the meta walk is recursive. |
| Secret, secretValue(field) | The type of every config field that holds a credential (databaseUrl, secretKey, redactionSecretKey, every apiKey, neo4jPassword). You pass plain strings; read one back with .get() or secretValue(field), which also takes a plain string or null. JSON.stringify, util.inspect, console.log, String() and a spread show "**********". See Secret fields. |
| redactionKey(config), checkRedactionKey(config) | The effective hash key (redactionSecretKey, else secretKey), and a check that throws when a hash rule has no key. |
| graphNeo4jConfigured(graph) | Whether a Neo4j mirror is configured (neo4jUri is set). Without one, graph mode runs on Postgres alone. |
| functionTool(fn), resolveApproval(engine, id, decision, approver, meta?, { principals? }) | The schema a function tool gets, and resolving an approval to "approved" or "rejected". |
| computeOverFrames(frames, instruction, { config, frameColumns?, model?, codeRunner?, ... }) | compute over frames you already hold, with the same guards and result shape. frameColumns maps a frame key to its column names, so an empty frame (a row array with no rows to read a header from) still names its header in the schema the model reads. model beats modelCfg, which beats config.llm. codeRunner runs the generated code out of process; the same static safety check runs before a host runner as before the in-process sandbox. |
| requestKey(request), embeddingRequestKey(text, { kind, model }) | The stable batch keys, identical in both ports and on every resume: sha256 hex over the canonical JSON of the request (text purposes post-redaction, vision over the page image bytes), and the per-text embedding key with model your config.embedding.model. See Batch mode. |
| formatResult(result, responseMode?), rowsToTsv(rows) | The same JSON or TSV shaping executeTool applies, for data you already hold. |
| LEXICAL_STOPWORDS | The built-in English stopword list search.lexicalMatch: "any" drops from a query when search.lexicalStopwords is unset. |
Presidio detectors
Presidio has no JavaScript port, so /presidio calls a Presidio Analyzer over HTTP at CE_PRESIDIO_URL (or PRESIDIO_URL), posting to /analyze with a blocking curl so redaction stays synchronous. With no URL set, building the detectors throws ExtraMissingError. An analyzer failure returns no spans (fail open). The default score threshold is 0.5 and the default language en.
import { RedactionPolicy } from "@promptev/context-engine";
import { presidioDetectors } from "@promptev/context-engine/presidio";
// CE_PRESIDIO_URL=http://localhost:5002 (a Presidio Analyzer)
const policy = new RedactionPolicy({
customDetectors: presidioDetectors(["PERSON", "LOCATION"], { language: "en", scoreThreshold: 0.5 }),
rules: [
{ name: "person", detector: "PERSON" },
{ name: "location", detector: "LOCATION" },
],
});
Exported types
Type-only exports from the package root, for annotating your own code. This page documents 36 engine methods.
| Types | What they describe |
|---|
| ContextEngineConfigInit, GraphConfigInit | What new ContextEngineConfig(init) and its graph section accept: plain strings where the built config holds a Secret. |
| EmbeddingConfig, LLMConfig, GraphConfig, RerankerConfig, FusionConfig, SearchConfig, SearchConfigInit, StorageConfig, ExtractionConfig | The config sections. |
| IngestReport, DocumentReport | What ingest and resumeEmbeddings return. A parked document’s status is "waiting_model"; abandonedRequestKeys lists the keys a pass deferred before it failed outright. graphBackendStale is true when a configured Neo4j could not be given the document’s graph: the document still completed, search works from Postgres, and resyncGraph() replays it. |
| ResumeReport, SupersededPark, ParkedDocument, ContentSource | What resumeDocuments returns and what its contentSource callback is called with. |
| ModelRequest, ModelReply, Purpose | What a host model receives and returns. ModelReply carries text, tokens, mode ("prompt" by default) and refusal. |
| RetrievalLeg, RetrievalScope, FusionFn, GraphBackend | The plugin seams: a leg’s signature and the frozen scope it receives, the fusion signature, and the graph backend interface. BUILTIN_LEGS (exported) holds the reserved leg names, and RetrievalFailed is thrown when every leg failed. SearchResult.timings holds each leg’s wall time in milliseconds, never part of usage. |
| SearchResult, Hit, GraphReport, GraphReason | Search output. A Hit is { documentId, documentName, description, sourceId, chunkText, idx, lang, score, chunkId, meta }; score is the fusion score even when a reranker decided the order. |
| UsageEvent | { kind: "ingest" | "search" | "tool", units, detail?, providerTokens? } |
| Hooks, ProgressEvent | See Hooks. |
| Principals, Trusted, Unset | Sentinel-aware types for principals and PATCH fields. |
| Scope, ScopeInput, KnowledgeAction, KnowledgeComputeFn | The knowledge tool’s ceiling and host-supplied compute. |
| ToolKind, ApprovalRecord, ResponseMode | "http" | "db" | "mcp" | "function", a stored approval, and "json" | "tsv". |
| CodeRunner, TaskRunner, TaskStatus | Host seams for code execution and task submission. A CodeRunner is called with (code, context, { timeout }); its result is trusted only as far as JSON. The static safety check the engine runs before it is internal, not an export. |
| FetchFactory, FetchPurpose, HostLookup | Host seams for outbound HTTP and DNS. |
| JsonSchema, SchemaName, StructuredCallOpts, StructuredResult, Mode, StructuredReply | Structured-output helpers. |
| ChunkRow, SearchScope, StorageBackend, ExtractionResult, QueryType, EmbedKind, RedactionRuleInit, ComputeDocument, ComputeFrames | Lower-level building blocks. |