What is private AI?

Private AI is about control. When you use an AI assistant or agent, your prompts, documents and tool results travel to a model and are often stored along the way. Private AI means you decide where that happens, who can see it, how long it is kept and whether anyone trains on it.

There is no single product called private AI. It is a set of choices, and each one trades convenience for control.

Why do companies want private AI?

• Confidentiality. Contracts, source code, financials and customer data should not end up with an unvetted third party.
• Compliance and data residency. Some data must stay in a given country, cloud or building.
• Training concerns. Companies want a clear commitment that their data will not train someone else’s model.
• Customer requirements. Enterprise customers ask vendors where their data goes, and expect a precise answer.
• Cost and choice. Owning the model relationship lets you pick providers per task and pay them directly.

What are the levels of private AI?

Think of it as a spectrum. Each step to the right gives you more control and asks more of you.

Level	Where the model runs	Where your data is stored	Who has the model contract	Effort to run	Main trade-off
Public chatbot (consumer plan)	Provider’s cloud	Provider	The individual	None	Least control; terms may allow training
Enterprise plan of a public AI service	Provider’s cloud	Provider, under a business agreement	Your company	Low	Stronger terms, but data and history sit with the vendor
BYOK platform	The model provider you choose	The platform, plus the provider’s retention	Your company, directly with the provider	Low to medium	Clear model terms and choice; the platform still holds data
Platform deployed in your own cloud or on premises	The provider you choose, or your own models	Your own environment	Your company	Medium to high	Data stays with you; you operate the deployment
Self-hosted open models	Your own GPUs	Your own environment	Nobody outside	High	Nothing leaves; you own model quality, capacity and updates

What is a self-hosted LLM?

A self-hosted LLM (large language model) is an open-weight model that you run on your own servers or cloud account instead of calling a provider’s API. Popular serving software such as vLLM and Ollama can expose an OpenAI-compatible API, which means many AI apps can talk to your own model the same way they talk to a hosted one.

Benefits:

• Prompts and documents never leave your environment.
• You control the model version, so behaviour does not change unexpectedly.
• No per-token bill to an outside provider, though you pay for hardware.

Costs and limits:

• Hardware. Larger models need expensive GPUs to answer quickly.
• Operations. Someone must deploy, scale, patch, monitor and upgrade the model.
• Quality. The strongest hosted models are often ahead on complex reasoning and long agent tasks.
• Features. Tool calling, long context and file reading vary between open models and serving setups. Check that the model supports tool calling before using it for an agent.

What is BYOK AI?

BYOK (bring your own key) means the AI platform calls a model provider using your API key instead of its own. It is often the most practical middle ground.

• You hold the contract with the model provider, so its business data terms apply to your account. For example, OpenAI states in its API data usage documentation that API data is not used for training unless you opt in.
• You see the costs directly on the provider’s bill.
• You can switch providers or use different models for different jobs.

What BYOK does not do: data still travels to the model provider. If some data must never leave your network, pair BYOK with your own deployment and a self-hosted model for that data.

On-premise LLM vs cloud AI: how do you choose?

Start from the data, not the technology.

1. Classify your data. Which classes are public, internal, confidential or regulated?
2. Map rules to each class. Which may go to a provider under contract, and which must stay inside?
3. Pick the lightest level that meets the strictest rule for each class. Many companies use BYOK for most work and a self-hosted model for a few sensitive workloads.
4. Test quality of candidate models on your own tasks before committing.
5. Count the running cost: hardware, people and upgrades, not just licences.
6. Keep the other controls. Private hosting does not replace permissions, masking, approvals and audit. See AI data leakage for the paths that remain.

The NIST AI Risk Management Framework is a useful neutral structure for documenting these decisions and their risks.

What private AI does not solve

• Oversharing inside the company. An assistant on your own servers can still show a salary file to the wrong employee if retrieval ignores permissions.
• Prompt injection. A self-hosted model can be manipulated just like a hosted one.
• Shadow AI. If the private option is slow or weak, people go back to public tools. See What is shadow AI?.

How promptev handles private AI

• promptev does not host or sell models for agents. Agents run on your own model keys (BYOK), from built-in providers such as OpenAI, Anthropic, Google Gemini and DeepSeek, or 34 ready-made providers, including Azure OpenAI, Google Vertex AI and Amazon Bedrock.
• Custom providers accept any OpenAI-, Anthropic- or Gemini-compatible endpoint, so a self-hosted model served that way can be used, with your own model prices.
• Each agent can use its own model, and the console blocks a model without tool calling for an agent with tools.
• promptev cloud runs on Google Cloud in the United States. On Enterprise, promptev can be deployed in your own cloud, in any region, or on premises. See deployment options.
• Your data is never used for training, data is encrypted at rest with AES-256, and promptev has completed SOC 2 Type II.
• Documents, embeddings and retrieval can point at your own PostgreSQL database with your own embedding model.