What I learned about cost, control, and practical AI by moving from a small test setup to a larger local cluster.
In the early days of agentic computing, I had an AI agent working on a project when it hit an error it could not get past. Without my realizing it, the agent kept trying to solve the same problem over and over. Each attempt sent more information to the model and generated another response, and every one of those was billable. In roughly two days, those repeated attempts ran up about $4,000 in API charges. Talk about having your heart sink.
That expensive lesson is part of why we put new technology through its paces before it goes anywhere near a customer's operation. We learn where it breaks and use those lessons to build better solutions. We make the early mistakes so our customers do not have to.
I have spent 35 years helping independent insurance agencies with their technology, and my interest in AI goes back further, to the expert systems of the late 1980s. Software that could help people solve problems and make decisions was an exciting idea that never became the everyday business tool I imagined. Transformer-based AI and ChatGPT changed that.[1] SMART Services has now built and is running its own local AI cluster in our own office, doing real work for our own AI agents. As far as we know, we are one of the very few technology partners serving independent agencies that has done this. Here is what we have learned so far.
The bottom line: You do not need to buy anything today. Start with one frequent, measurable task on an approved AI tool, and get your data, permissions, and workflows organized. Running AI in-house is becoming practical, and for many agencies it is a year or two away. SMART Services is already running it, and we are learning what it takes so you will not have to.
Before getting into the cluster, one distinction matters more than any other, because it drives the cost of everything that follows.
Chatting with AI and putting agentic AI to work are not the same thing – technically or financially.
AI Agents Change the Meter
When I talk about an AI agent, I mean AI software that can work through a job using the tools and access we give it. It might read a service email, look up information, draft an activity for the agency management system, and prepare a reply for an employee to review. The model supplies the reasoning; the agent handles the steps. Throughout this article, an AI agent is software, not a licensed insurance agent.
Token: A small piece of information an AI model reads or produces, often a word or part of a word. Providers count tokens to measure usage and calculate charges.
That is powerful, but it changes the meter. AI agents can work around the clock and start tasks on their own. They consume tokens as they work, so one busy AI agent can use far more than a person having an occasional chat.
Providers charge for that usage in two ways. Subscriptions such as Claude's Max plans or Microsoft's Copilot include a set amount of usage, with limits. A heavy project can use it up early and leave you waiting for a reset in the middle of the workday, unless you pay for more.[2][3] The other way is an API, a direct connection between software systems, where every turn goes on the meter. The price per token keeps falling, but the amount we use is growing faster.[4] That is how my $4,000 agent happened.
It is also why I prefer subscription-based access for most of our agent work. As long as the work stays within the included usage, I am not paying a separate charge for every additional token, and the bill is far more predictable.[5] Either way, spending limits, alerts, and a stopping rule that brings in a person after repeated failures all belong in the setup.
The scale can become startling. A publicly shared OpenAI dashboard from OpenClaw, an open-source platform for running AI agents, showed about $1.3 million in API-equivalent usage in one month from roughly 100 AI agents. Uber offers a named business example: in April 2026, its CTO said the company had already used up its full-year AI-tools budget after roughly four months of rapidly expanding use. It later stabilized spending with tighter limits, better cost visibility, smarter model routing, and greater use of lower-cost and open-weight models.[6]
None of that means frontier AI is a bad idea. It means we need to manage agentic AI like a real operating expense.
Agentic AI needs financial discipline: a budget, an owner, alerts, and a reliable off switch.
One more thing to keep in mind: today's flat-price plans for heavy users are subsidized by the investor capital financing enormous model and computing build-outs, and the companies behind them are moving toward the public markets.[7][8] I expect shareholder pressure to show up eventually – in prices, limits, metering, model access, or efficiency.
I believe AI tokens will become a business utility.
I see AI tokens becoming much like electricity, natural gas, and internet access. When we first approached agencies about installing internet access, more than one owner told me they would be dead before it went into their office. Well, they are still here, and it would be difficult to find an agency operating without internet today. AI adoption is headed in the same direction, only much faster.
Once I started looking at tokens as a utility, the obvious question was: does every repeated agency task need to run through somebody else's meter? The answer is no.
Figure 1. Token prices are falling while token usage is growing. The price line is a broad index; worldwide usage is a modeled estimate.
Why I Started Looking at Local AI
Running a model locally gives us more control over where agency information is processed, who can access it, and how the system is backed up. Work can stay inside the agency's office or a private environment managed by its technology provider. And once the equipment is in place, another local AI task does not add a charge to a frontier provider's token bill.
Open-weight model: An AI model you can download, install, and run on your own equipment, instead of relying on a hosted commercial service such as ChatGPT or Claude.
The early open-weight models were too slow and not smart enough for serious agency work. The gap has narrowed quickly. People may recognize names such as Meta's Llama, Mistral, Alibaba's Qwen, DeepSeek, and Moonshot AI's Kimi, and each generation has moved closer to the frontier models on reasoning and tool use. For a frequently updated comparison, see ModelCap's Open-Weight LLM Rankings.[9]
AT&T and Microsoft report that open-weight models processed about a trillion tokens for AT&T's telecom-model program and saved tens of millions of dollars compared with frontier models.[10] That is large-enterprise evidence, not proof that every agency should install a model tomorrow, but it shows open-weight models are no longer a hobbyist experiment. An agency does not need AI to solve a decades-old mathematical problem. It needs dependable help with repeated transactions, service work, renewals, documents, and follow-ups.
AI tools: Connections and functions that let an AI agent take action, such as reading email, creating a Word document, using a browser, or working with an agency management system.
Those connections give the AI agent an approved way to request information or use another application, such as a rating system or agency management system. Access still needs limits. An AI agent might gather information, prepare an activity, or draft a client email, but a qualified person should review important actions and decide what gets posted, sent, quoted, or bound. Keeping a person responsible for those decisions is what we mean by a "human in the loop."
Test scores and demonstrations are interesting, but an agency owner eventually asks a simpler question: can my people depend on this during a real workday? We learned the answer by building it.
What Our First Cluster Taught Us
We started with two ASUS Ascent GX10 systems, small desktop-sized AI computers, connected directly by one high-speed cable. One ran approximately 10 to 15 AI agents. The other hosted a smaller local model devoted largely to memory embeddings, the searchable index that lets an AI agent find relevant information in earlier documents and conversations even when the wording differs. Building and updating that index takes processing, and moving it onto a small local model reduced the work going through our paid frontier accounts.
Figure 2. SMART Services' first local AI setup. The two ASUS GX10s were linked directly by a high-speed cable; one ran roughly 10 to 15 AI agents while the second handled local memory embeddings and support work.
That first step worked, and it taught us a great deal. The AI agents could handle useful work, and the local model reduced some of their token use. They still depended on frontier models for harder reasoning and tool use, though. We wanted to bring more of that work in-house and make the system dependable enough for everyday office workflows. That first cluster changed the question.
The real question is not whether AI can run locally. It is whether an agency can depend on it on Monday morning.
Why I Built the Larger AI Server Cluster
Next, I wanted a larger local model that could do more of the reasoning for our AI agents and use the tools needed for real office work. That called for much more memory and computing power. One option was a system based on NVIDIA's DGX Station GB300, a powerful AI workstation built for exactly this kind of work. Systems in that class run roughly $100,000.[11][12] That was well beyond the budget I had in mind.
After more than 30 years of building and supporting business servers, I wanted to see how close we could get for less money. I spent a great deal of time researching the models, the equipment they required, and ways to connect smaller systems so they could work together.
AI server cluster: Several computers connected to work together. The software can spread a larger AI model across their combined memory and processing resources.
That research led us to four ASUS Ascent GX10 systems powered by NVIDIA's GB10 Grace Blackwell platform. The real breakthrough was finding a very high-speed switch that connects all four servers together. How the systems are connected turned out to matter as much as the systems themselves. That part took real research, and it is know-how we now carry into every cluster we build.
Our four GX10 systems, switch, and cabling cost about $17,591, less than one-fifth of the cost of one DGX Station-class system. Our cluster is not as powerful, but it gets us much closer to the capacity we need at a price we could justify. Together, the four systems give us 512 gigabytes of memory, roughly what you would find in 32 typical desktop computers combined, along with the fast connections that let them share the work while a model runs.[11]
Figure 3. The four-system GX10 cluster that followed. Each system connects to the high-speed switch; together they provide 512 gigabytes of memory for running larger open-weight models.
We tested NVIDIA Nemotron 3 Ultra, Moonshot AI's Kimi K2, and other models along the way, and eventually settled on Ornith, an open-weight model released by DeepReinforce. Just before finishing this article, Ornith 1.5 became available. It is much more capable and about the same size, so it runs on the equipment we had already built, using about 450 to 470 of our 512 gigabytes of memory.[13] We had one of our AI agents handle the upgrade. Within a couple of hours, it reported that the upgrade was complete, its testing and validation checks had passed, and the system was ready to use. What we are having agents do behind the scenes could be an article of its own.
Our results with Ornith are promising. It responds quickly and uses tools well. We are still tuning reliability over longer jobs, but it is getting very close to the level needed for practical commercial work. The upgrade also illustrates something important: better models make existing equipment more useful. New models arrive almost daily, and we do not switch for each one. We move when the improvement is worth the testing and setup.
What It Costs to Build – and Operate
That brings us to the question every business owner eventually asks: what does it cost? I want to answer it one way, over three years, so the two options can be compared side by side.
Using the prices from when we planned the project, our four-system build came to approximately $17,591, including the GX10 systems, high-speed switch, and cabling. Hardware prices move quickly, so any new project should use current quotes.[11][14] Running it costs about $870 a year in electricity. Someone also has to look after it, about five hours a month in our case, which we handle as part of the work we already do. For comparison, the four-account agent stack we use – two $200 OpenAI accounts, one $100 Grok account, and one $20 Ollama account – costs $520 per month.
Figure 4. Three-year cost comparison. The subscription stack runs about $18,700 over 36 months. The local cluster, equipment plus three years of electricity, runs about $20,200.
The reason to build local is not to save money today. It is control over where your information goes, capacity that does not run out in the middle of the afternoon, and less dependence on somebody else's meter. And unlike a subscription, the equipment keeps getting more useful as better models arrive.
Nor is this quite apples to apples. With a subscription, the provider operates the AI system; with a local setup, we operate it and control its capacity. I do not expect local AI to replace frontier models, but I do expect it to offset a meaningful amount of their use.
Local AI Is Getting Closer
Open-weight models are getting faster and smarter. They are also becoming more efficient, requiring less computing power, and simpler to deploy and manage. That is why I believe practical local options for many more agencies could arrive in about a year or so, and that may be conservative.
One thing holding back the economics today is memory. Large open-weight models need a great deal of it, and tight supplies are keeping prices high; the industry outlook cited here points to pressure through 2027, with supply improving gradually in 2028.[15] Over time, I expect hardware to deliver more capability for the money and more efficient models to make better use of the equipment we already have. That combination could make larger local AI systems much more practical for small businesses within two or three years.
What It Takes to Run It
We often hear that an AI agent works 24/7, never takes a break, and never goes on vacation. It still runs on computer hardware that needs updates, monitoring, backups, security, and recovery. AI agents can lose their place, stop when another system fails, or simply run amok. They need what I call "care and feeding." Even our own cluster had an early problem: one job got stuck and ran at about 98 percent capacity for a day before I caught it. Compared with my $4,000 lesson, that one cost us almost nothing – a slightly higher electricity bill and maybe a bruised ego. I ended up using OpenAI's Codex to build a monitoring system so we catch problems like that sooner.
Figure 5. The live monitoring dashboard we built with OpenAI's Codex. Each of the four systems in the cluster shows its CPU utilization, its GPU utilization in the circular gauge, and its temperature. Temperature matters as much as the load.
Our cluster runs Linux. After spending roughly 30 years as a Windows guy, I had to become fairly capable in Linux very quickly to pull this build off. An agency does not need that expertise on staff, but it does need a technology partner that can design, secure, maintain, and recover the environment. Our system runs in our local office, and remote offices reach it through our protected business network rather than through a public doorway on the internet.
Technical readiness is only half the job. Governance matters just as much. Keeping a model local can give an agency more control over its customer information and its own operational data: procedures, workflows, internal knowledge, and business records. That includes the information it gives its AI agents and the answers they produce, which can reveal a great deal about how the agency operates.
Data sovereignty: I use this term to mean keeping the agency in control of where its information lives, how it is processed and used, and who can access it.
That control does not complete the compliance job. Michigan DIFS has told insurers to govern and document their AI, test it for errors and unfair bias, protect nonpublic consumer information, monitor performance, and scrutinize outside vendors. That bulletin applies directly to carriers and other certificate-holding insurers rather than ordinary independent agencies, but it is a useful signal of the standard agencies and their technology partners should expect.[16]
Some of this capability will arrive through software agencies already use. Agency management systems will absorb more AI, much as they absorbed document management. That will reduce the need for some outside AI tools, but it will not eliminate workflows that cross multiple systems or reflect the agency's own processes. So what should an agency do while the technology keeps improving? Start with the workflow, not the AI or the hardware.
Start With the Right Agency Problem
At one agency, a SMART associate asked employees to suggest a first AI use case. One employee named a task she hated because it took four hours. It sounded promising until we asked how often she performed it: once a year.
Saving four hours annually will not justify a complicated automation. A better target may take only five minutes but occur 20 or 30 times every day, especially when several employees perform it. Repetition, not how much somebody dislikes a task, builds the business case.
Good early candidates include routing service emails, preparing internal renewal summaries from defined sources, cleaning notes, flagging missing information, drafting an activity for the agency management system, or preparing a routine follow-up for human review. Poor first candidates include unsupervised client communication, coverage decisions, binding authority, and unrestricted access to the management system.
Model-agnostic agent: An AI agent that can use different models. You can change the model behind it while keeping much of the agent's tools and workflow the same.
The frontier providers are building their own AI agents, but agencies have another option: agent platforms that work with different models. An agency can begin with an approved frontier service such as ChatGPT or Claude, then move suitable work to a capable local open-weight model while keeping the rest the same. Most of my AI agents use a frontier model first, with an in-house model as backup when they hit a usage limit, and in our testing some workloads now run on an open-weight model full time. A larger local deployment deserves a close look when the local models are capable enough, the agency has learned how to manage AI, and token cost, data control, or workload scale has become important.
Wherever an agency is in the process, the most important step is an agency AI roadmap: organize the data and put sensitive information in the right silos; align security, permissions, and access rights; document workflows and choose frequent, measurable automation candidates; and involve, motivate, and train the people who will use and supervise the system. The agencies that benefit most will not have the biggest model or the most AI agents. They will have organized data, sound access controls, documented processes, trained people, a responsible person in charge, and a clear business problem worth solving, whether their AI runs on a frontier model, a local model, or both.
Build the foundation first. Then put the right AI on the right problem.
Where SMART Services Fits
I did not build this cluster to argue that every agency needs one. I built it to answer a practical question for our customers: can local AI become capable and dependable enough to do real agency work? Based on what we have seen, the answer is yes. It still needs tuning, but the gap is closing fast, and we intend to be ready when it closes.
If you would like to see the cluster, talk through a first AI task for your agency, or get started on the roadmap above, contact us at contact SMART Services. Agency owners can start today with an approved model and a frequent, measurable task. As local options improve, we will help you expand what works and decide when the costs and benefits justify bringing more of the work in-house.
Sources / Further Reading
[1] Vaswani and colleagues, "Attention Is All You Need" (2017), the paper introducing the Transformer architecture. [Research paper]
[2] Anthropic, "What is the Max plan?" and "Manage usage credits for paid Claude plans." [support.claude.com] and [support.claude.com]
[3] Microsoft, "Copilot Cowork is now generally available" and "Copilot Credits." [microsoft.com] and [microsoft.com]
[4] OpenAI, GPT-5.6 Sol and legacy GPT-4 model pricing. [developers.openai.com] and [developers.openai.com]
[5] OpenAI, "ChatGPT Work and Codex Pricing" and "Authentication." [learn.chatgpt.com] and [learn.chatgpt.com]
[6] Tom's Hardware, "OpenClaw creator burned through $1.3 million in OpenAI API tokens in a single month"; TechCrunch, "Uber caps employee AI spending after blowing through budget in 4 months"; Axios, "Uber cuts AI costs even as usage jumps." [tomshardware.com] and [techcrunch.com] and [axios.com]
[7] Axios, "AI companies cover costs – but not forever"; OpenAI and Anthropic funding announcements. [axios.com] and [openai.com] and [anthropic.com]
[8] OpenAI and Anthropic confidential S-1 announcements; SpaceX Form 10-Q for the quarter ended June 30, 2026. [openai.com] and [anthropic.com] and [sec.gov]
[9] Meta, Llama 2; Mistral AI, Mistral 7B; Qwen, Qwen2.5; DeepSeek, DeepSeek-R1; Moonshot AI, Kimi K2 and Kimi K3. [ai.meta.com] and [mistral.ai] and [qwenlm.github.io] and [deepseek.com] and [moonshotai.github.io] and [huggingface.co]
[10] Microsoft and AT&T, commercial open-model deployment; Microsoft, local AI on Windows. [azure.microsoft.com] and [about.att.com] and [developer.microsoft.com]
[11] NVIDIA and ASUS, GX10 clustering and 200-gigabit networking; NVIDIA, DGX Station specifications. [docs.nvidia.com] and [docs.nvidia.com] and [nvidia.com]
[12] Exxact, Valence VWS-158270643 GB300 AI workstation, published starting price $94,930, checked September 8, 2026. [Product and pricing]
[13] Ornith, Ornith 1.5 release announcement, model card, and model files. [Release announcement] [Model card] [Model files]
[14] ASUS, GX10 specifications and purchase links; U.S. Energy Information Administration, June 2026 commercial electricity update. [asus.com] and [eia.gov]
[15] TrendForce, 2027 DRAM outlook; Micron, fiscal Q3 2026 prepared remarks. [trendforce.com] and [investors.micron.com]
[16] Michigan Department of Insurance and Financial Services, Bulletin 2024-20-INS, "Use of Artificial Intelligence Systems in Insurance." [michigan.gov]
This article is educational and is not legal, regulatory, coverage, investment, or financial advice. Agency requirements and AI service prices change. Verify current terms and consult appropriate legal, compliance, carrier, cybersecurity, and financial advisors before acting.
About Jerry Fetty
Jerry Fetty is the Founder of SMART Services and has spent 35+ years helping independent insurance agencies modernize their technology, strengthen cybersecurity, and operate more efficiently. Today, his focus is helping agencies adopt AI the right way, with a secure foundation, clean data structure, clear policies, and real-world training that produces measurable ROI. He designed and operates SMART Services' own local AI cluster, which runs the company's AI agents on open-weight models.