Air-Gapped Deployment & Local LLMs (vLLM / Ollama)
AIST is designed from the ground up to operate inside completely isolated enterprise intranets where neither the server nor client hosts have outbound internet access.
Why AIST Works Offline Out of the Box
- Bundled Client Distribution:
ghcr.io/llaoj/aist-serverembeds all 7 Linux/macOS client.tar.gzarchives andchecksums.txtinside/usr/share/aist/client(~28 MB compressed, ~42 MB total image). - Self-Contained Bootstrap Script: When an internal server runs
curl -fsSL https://aist.corp.internal/run.sh | sh,run.shdownloadsaistdirectly fromhttps://aist.corp.internal/releases/client/— zero calls to GitHub, CDNjs, or external registries. - Strict Self-Hosted Web Assets: All HTML, CSS, SVG logos, and JS in
/and/registerare embedded in theserverGo binary.
Step 1: Import the Server Image into Your Air-Gapped Network
On an internet-connected workstation:
docker pull --platform linux/amd64 ghcr.io/llaoj/aist-server:0.6.0
docker save ghcr.io/llaoj/aist-server:0.6.0 | gzip > aist-server-0.6.0-amd64.tar.gzTransfer aist-server-0.6.0-amd64.tar.gz into your internal network and load it:
docker load -i aist-server-0.6.0-amd64.tar.gzStart with Docker Compose (using internal TLS certificates on your reverse proxy, or trusted internal CA installed on client nodes) and set Settings → Application → Application URL to your internal address (e.g. https://aist.corp.internal).
Step 2: Connect vLLM or Ollama as an Upstream Provider
Any OpenAI-compatible /chat/completions endpoint that supports function/tool calling (tools array) works with AIST.
IMPORTANT
Choose models with reliable tool-calling capabilities (for example Qwen2.5/Qwen3 32B+, DeepSeek-V3/R1-Distill 32B+, or Llama 3.3 70B). Small models below ~14B often hallucinate shell JSON arguments or misclassify query vs modify commands.
vLLM Configuration
Start vLLM with tool-call parsing enabled (example with Qwen):
vllm serve Qwen/Qwen2.5-32B-Instruct \
--enable-auto-tool-choice \
--tool-call-parser hermes \
--max-model-len 65536 \
--port 8000In AIST Admin (/_/):
upstream_providers:name:intranet-vllmbase_url:http://192.0.2.50:8000/v1api_key: leave empty if unauthenticated, or enter your vLLM--api-keyenabled:true
upstream_models:provider:intranet-vllmname:Qwen/Qwen2.5-32B-Instructdisplay_name:Qwen 2.5 32B (Intranet)priority:10context_window:65536max_output_tokens:8192enabled:true
Ollama Configuration
Expose Ollama to your AIST server container and create:
upstream_providers:name:local-ollamabase_url:http://192.0.2.51:11434/v1api_key: (leave empty — AIST omits theAuthorizationheader whenapi_keyis empty)enabled:true
upstream_models:name: e.g.qwen2.5:32bcontext_window: matchnum_ctxconfigured on your Ollama Modelfile
Step 3: Install Clients Across Internal Jump Hosts
On any Linux or macOS machine inside the intranet:
curl -fsSL https://aist.corp.internal/run.sh | shData never leaves your private network.