Aspirational — not yet shipped. This page describes a target requirement, not a delivered capability. The dedicated local OpenAI-compatible provider it assumes is not verified in the current build, and no on-prem overlay ships it. Treat everything below as the contract a future local-LLM deployment must meet, and confirm provider support in your target release before planning against it.
Model-serving requirements
- The model is pinned by exact artifact version, image digest, and serving configuration.
- The serving endpoint is internal-only.
- Readiness must mean the model is loaded and able to answer a small chat request.
- Health checks must fail when the model is unloaded, the GPU is unavailable, or the inference runtime cannot allocate the required memory.
- The model must support reliable structured tool/function calls with the tool schemas that Aperium sends for every enabled MCP connector.
- Context length must be sufficient for the system prompt, conversation context, and the active MCP tool schema set.
- GPU memory, batching, concurrency, and max-token settings must be sized from load tests, not defaults.
Required env profile
Use the dedicated local OpenAI-compatible provider with an internal base_url. These values map to the LLM providers section of the env reference:
Do not configure ENABLE_LLM_FALLBACK=true to a cloud provider for this deployment shape unless your security model explicitly permits data leaving your network boundary.
Why fallback is off
The on-prem deployment shape exists to keep inference inside your network boundary. Re-enabling cloud fallback silently violates that contract. Treat any change to ENABLE_LLM_FALLBACK as a security review item: a release that flips it on must be accompanied by an explicit, documented exception from your security model.