Frequently Asked Questions: Local AI Agents on Your Own Hardware

That depends on model size and the number of users. For models up to roughly 120 billion parameters in 4-bit quantisation, devices with 128 GB of unified memory are enough, such as an NVIDIA DGX Spark or a machine with AMD Ryzen AI Max+ 395. Larger models need a Mac Studio with M5 Ultra (up to 512 GB; these configurations ship from late October 2026) or several networked devices. In the workshop we decide which class fits your tasks.

For many business tasks – summarising, searching your own documents, drafting, classifying, coding – current open models are good enough. For very complex tasks the large cloud models are often still ahead. That is why we test the models in advance with your own data, including German-language data, instead of relying on benchmarks.

The model's processing runs entirely on your hardware. Whether an agent may also access the internet, for example for a web search, is your decision. Fully air-gapped operation without an internet connection is possible but requires additional hardening for installation and updates – we can plan that in on request.

Hermes Agent is an open-source agent framework by Nous Research under the MIT licence. It works with any local model server, comes with memory, skills, a scheduler and connections to Slack, Telegram or email, and connects your systems via MCP. Because it evolves quickly, a managed update process is part of our handover.

Run locally, these models send no data to their vendor – the model weights are files on your hardware. Many variants are licensed under Apache 2.0 or MIT and can be used commercially; we check the licence for each model. Like any model they need guardrails for their behaviour, and for customer-facing applications we choose model and safeguards with particular care.

Yes. Which obligations apply depends on the purpose, not on where the model runs. Running locally does make control and documentation easier, because you know which model in which version processes which data.