Running OpenAnt Fully Local: The Ollama Adapter

Posted by Kris Racette on August 25, 2026 · 8 min read · Security Achievements

How we extended OpenAnt so SAST scans run against local models — no cloud, no API keys, no source code leaving the machine.

Why this matters

Static application security testing (SAST) tools increasingly use LLMs to cut false positives and explain findings. The catch: most integrations assume a cloud LLM. That means your source code — sometimes your crown jewels — gets shipped to a third-party API to be analysed for vulnerabilities.

For security-conscious teams that's a non-starter. Air-gapped environments, regulated data, client confidentiality agreements, or just plain principle: the code shouldn't leave the box.

So we built a first-class Ollama adapter for OpenAnt.

What we built

• Provider adapter (openant-core/utilities/llm/providers/ollama.py): a thin delegation over OpenAnt's existing OpenAI-compatible helpers. Ollama exposes an OpenAI-compatible endpoint at http://localhost:11434/v1, so the adapter slots into OpenAnt's provider architecture cleanly. API key is optional (it's local), and configuration is fully supported in the setup wizard.
• Setup wizard integration: Ollama appears as a first-class provider choice, with connectivity probing and sensible model defaults per tier (e.g. Qwen3.8 — current gen).
• Model registry entries in models.json.
• Full test coverage: a contract-test factory plus a dedicated adapter test suite, consistent with how every other provider is tested upstream.
• Documentation (OLLAMA.md) covering install, model selection, and warm-up behaviour.

Everything is Apache 2.0, matching upstream, and structured as an upstream contribution rather than a fork.

Verified end-to-end

We validated the whole pipeline on a developer workstation:

• OpenAnt CLI installed, wizard configured with provider=ollama
• mistral-small3.2 (15 GB) running locally via Ollama
• Scanned a repository with planted vulnerabilities
• Both planted SQL injection findings (CWE-89) detected correctly — with zero network egress beyond the local machine

No cloud calls. No keys leaving the box. The scan results were accurate against a local model on commodity hardware.

Practical notes from real usage

Things we learned that aren't in the docs:

• Use chat models. Base completion models reject /chat/completions endpoints — pick instruct/chat variants.
• Mind cold loads. Large models (15 GB+) take several minutes to load on first inference. Warm up before demos or CI runs. Very large models can wedge slower machines entirely — size your model to your hardware.
• Probe timeouts: slow cold starts can exceed default probe timeouts — worth documenting or bumping for big local models.

Where this fits

This is the kind of work we build for clients who can't ship their code to a vendor cloud — defence, healthcare, finance, legal, infrastructure operators, and any team with a hardened perimeter. Privacy-first security tooling isn't a marketing line for us; it's the default.

If you need SAST, code analysis, or any LLM-assisted security tooling that runs on your hardware and nowhere else, get in touch. The same architecture extends to dependency review, secret scanning, threat-modelling copilots, and code-review agents.

What's next

• Upstream PR to OpenAnt so every user gets native Ollama support — pending, link goes live once merged
• More local-model benchmarks across finding precision/recall
• Deployment guides for air-gapped and regulated environments

Fully local, LLM-assisted SAST is practical today. Your codebase stays yours.