In short: Step by step, individual AI experiments have turned into a small infrastructure: an AI server for the heavy computing, a mini PC that controls workflows, a NAS for data and backups and a web server for websites and social media. Each device has a clear job. The key lessons: separate computing from control, let scripts deliver the facts and don't run more models than you really need.
The devices and their jobs
AI server: ASUS Ascent GX10
The ASUS Ascent GX10 with 128 GB of shared memory is the computing base. It only does what needs a lot of power:
- Language models via Ollama: a large Qwen model for my agent and Gemma 4 31B for German texts
- vLLM for bulk jobs: when thousands of product descriptions are needed, vLLM processes many requests in parallel. How much that helps is shown in my test of local AI models.
- Open WebUI as a chat interface in the browser
- ComfyUI for images, only with Apache 2.0 licensed models and only behind password-protected access
- Rendering the quiz videos
The central schedules and automation state live on the control machine. The AI server still holds its own service configuration and model files, but does not own workflow scheduling.
Control machine: GEEKOM MiniAir 11
The small mini PC runs around the clock and controls everything that should happen regularly. It runs the agent Hermes with:
- watchers for Shopware news, freelance projects and my websites
- the weekly workflow for the quiz shorts
- the pipeline for the Shopware news videos
All notifications and approvals go through Telegram.
NAS
The NAS holds data that should stay: backups of my web server, the archive of all videos with sources and scripts, my documents in Paperless-ngx and my photos in Immich.
Web server
My web server runs my websites and Postiz, which I use to schedule posts for YouTube, LinkedIn, Instagram and other networks.
Laptop
The laptop, a TUXEDO notebook running Linux, is where I develop. I work with AI coding assistants there, test in Docker and review every change before it is committed. I'm currently testing local models directly in the editor, for example with Zed.
How it all fits together
- Mini PC → AI server: Hermes calls the models on the GX10 through the local API and has videos rendered there.
- NAS: the mini PC writes the video archives to the NAS, and the NAS pulls the web server backups itself.
- Mini PC → web server: finished posts and videos go out to the social networks via Postiz.
- Laptop and phone: I reach everything via VPN, notifications and approvals come via Telegram.
What I learned building it
Separate computing from control. The powerful machine computes, the small one decides. That way I can restart or rebuild the AI server without losing any schedules.
Take fixed data from verifiable sources. Language models can alter links and invent dates. Scripts therefore take URLs and dates from designated sources, while AI helps with judgement and wording. Those scripts still need tests and cannot make incorrect source data true. Why that matters so much is explained in When Classic Automation Beats AI.
Memory is the scarcest resource. Once, when starting a large model, I gave it too large a share of memory, and the AI server was unreachable for almost half an hour. Since then there are fixed upper limits, and large models are not loaded in parallel when memory gets tight.
Fewer models are better. After many tests I only run the models that have proven themselves. Five variants of the same model bring no benefit, just memory use and confusion.
Don't expose anything openly. The model APIs are only reachable on my own network, via VPN when I'm away. Every tool with a web interface gets password protection. And the automation gets its own restricted access, such as an SSH key that only allows rendering on the AI server.
Mind the licences. With image models in particular, not every model is cleared for business use. I only download models whose licence allows it.
Document everything. Models, integrations and tools change constantly. Without notes on what runs where and why, you quickly lose track.
Where I still see potential
I find the combination of local AI with recurring tasks particularly exciting: research, document organisation, code analysis and product data. For each use case I decide again whether automation really saves time or just adds maintenance.
Conclusion
My setup is no proof that everyone needs their own AI server. How to decide what should run locally and what in the cloud is covered in Local AI or Cloud. But it shows what becomes possible when models, tools and automation work together. The biggest gain is not a single spectacular answer, but a system that fits my work step by step.
Frequently asked questions
What does such an infrastructure cost?
The biggest item is the AI server. The mini PC and NAS cost much less, and I need the web server anyway. On top of that come power and, above all, time for setup and maintenance.
Can you start smaller?
Yes. A single machine with Ollama and a few scripts is enough to start. Splitting things across several devices only pays off once many workflows run permanently.
Does everything run locally for you?
No. I still use cloud AI for many tasks, for example when developing. What runs locally is anything with many requests, confidential data or that has to run around the clock.