An AI chatbot gives an answer to every question. An AI agent is far more autonomous and goes beyond simple answers; it makes plans, calls tools, checks its work, and repeats the loop until it’s finished. Little of that thinking reaches the screen, but it has a cost in tokens.
Dr. Jon Krohn, fellow of machine learning practice at Lightning AI, a cloud-based provider of AI development tools, explains the math: cost per token, multiplied by how much background thinking the agent does, multiplied by how many data points it creates. “All these multiples mean token usage is exploding,” Krohn says.
Total inference cost depends on the cost of processing each token, the number of tokens required per task, and the volume of tasks. Agentic workflows can increase all three, because agents plan, call tools, check results, and retry.
Generic models make generic agents
“If the model knows a little about everything, but nothing about your business, you’ll get sensible-looking answers, but not the best answers,” Krohn says.
An off-the-shelf model not fine-tuned to your business produces a generic agent. For a defined enterprise task, a smaller model trained on relevant data can be more efficient and accurate than a general-purpose model. What’s more, it costs less per token and processes far fewer, because it isn’t reasoning toward an answer it already knows.
Accuracy compounds the savings: A model that gets things right more often requires fewer people to check it. Small models keep GPUs busier, which means less wasteful downtime.
Krohn says the total benefit is not a few percentage points better — it’s orders of magnitude better. Gartner predicts that organizations will use small, task-specific models three times as often as general-purpose LLMs by 2027.
The obstacle is the patchwork most organizations manage, characterized by underpowered devices, scattered data, and disconnected systems. Dell’s full-stack AI approach with Nvidia fixes that with three components.
Deskside Agentic AI runs production-ready agents locally on enterprise hardware — where the token meter stops running. The AI Data Platform brings scattered enterprise data together into a foundation agents can actually use. And the AI Factory Foundation supplies validated architecture and tested deployment, so workloads don’t need rebuilding every time they grow.
From code to compute
Lightning AI runs its own data centers and sells elastic capacity, rather than reserved cloud instances, the fixed blocks that cost money even when nothing is running. “We’re fully integrated from the facility all the way to the application stack,” says Frank Basso, Lightning’s vice president of infrastructure.
The underlying hardware comes from the Dell AI Factory with Nvidia portfolio: liquid-cooled PowerEdge XE9780L and XE9785L servers built on Nvidia HGX B300, alongside Dell’s newest rack-scale systems built on NVIDIA GB300 NVL72. Lightning was among the first providers to deploy the GB300 racks, packing upward of 25 megawatts and 10,000 GPUs into a single 10,000-square-foot data hall — a footprint that would have required roughly 200,000 square feet in a conventional air-cooled data center. The liquid cooling runs in a closed loop, like a car radiator, so the data center consumes far less water.
Dell PowerRack arrives from Dell’s integration facilities already built and tested, and Dell field services certify them before handover, rather than Lightning assembling them on-site. For Krohn, that’s what makes the economics real: The Dell AI Factory with Nvidia “allows economic benefits of training our own AI models to go from theoretical to actually be implemented,” he says, because the cooling, GPUs, storage, and networking arrive as one kit.
Take steps that leading companies are doing differently
Krohn’s checklist for companies getting this right starts with understanding where efficiencies can be found. Model size and infrastructure efficiency can affect the cost of processing each token, while specialization can reduce the number of tokens, retries, and checks needed to produce a usable result. A winning strategy addresses both of these areas.
From there, start with evaluation, Krohn says. Know which workflow you want to change and how you’d measure whether a trained model improved it.
Next, treat training as a loop rather than an event, logging what goes wrong and feeding it into the next version. Finally, own the infrastructure.
With this sequence in place, Krohn and Basso agree, the payoff is both smarter models and exponential savings.
For high-volume inference, the potential savings come from combining smaller, task-specific models that use fewer tokens with an integrated infrastructure designed to keep computing resources productive and simplify deployment at scale.
Discover how Dell and Nvidia can help turn smaller, smarter AI models into lower-cost, production-ready agents.
This sponsored post was created by BI Studios with Dell AI Factory with Nvidia.
Explore other Dell AI Factory with Nvidia stories:
