If your software has an AI agent in it, most of what that agent does all day is probably dull. That's not an insult. It's the point of a new Gartner report, and it has real consequences for how you build and run an app that uses AI.
The Register covered it this morning. Gartner's report is called "Open-Weight Models Will Run 90% of Enterprise Agentic Workflow Steps", and the headline claim is simple: "Nine in 10 agent workflow steps are routine." Things like sorting an incoming email into a category, calling a tool, or checking that a bit of data is in the right shape. Gartner's view is that open-weight models (the ones you can download and run yourself, or rent from any number of hosts) handle those steps just as well as the big closed models, for far less.
What the numbers actually say
There are a few figures flying about, so it's worth being careful with them.
- At the same score on routine steps, Gartner puts an open-weight model at about a quarter of the price of the cheapest closed model.
- Against a frontier model used for every single step, it says the open model costs "as little as a twentieth".
- Then the honest bit. Open models "can be more verbose and retry more often", so the realistic saving against a frontier model is "closer to five or six times".
Five or six times cheaper is still a lot, and I like that the report says it plainly. The 90% figure is also a planning assumption for 2030, not a measurement of what's happening today.
The part I found most interesting wasn't the cost at all. Gartner says "Open-weights preserve your control. The cost saving is a bonus." It also points out that every routine step sent to a frontier API moves a slice of your workflow data outside your perimeter. For a business that cares where its customer data goes, that matters more than the bill.
The Register adds its own take: models are turning into commodities, and the value is in the harness, meaning the code that runs the agents, routes work to the right model and puts guardrails around it. That's the bit you actually build.
What this means if you've got an app with AI in it
Plenty of apps with AI features are built the simple way: pick a model, put its name in the code, send every call to it. There's nothing wrong with starting like that. But it means you can't take advantage of any of this without some work first. Here's how I'd go about it.
1. List every step the AI does
Write down each place your app calls a model. For each one, note what goes in, what comes out, and how you'd know if the answer was wrong. You'll usually find a split. A handful of steps need real reasoning (drafting a reply to an unhappy customer, working out a tricky quote). The rest are classification, extraction and formatting, the routine work Gartner is talking about.
2. Make the model a setting, not a line of code
If the model name is written into the code in fifteen places, changing it is a job. If each step reads its model from configuration, changing it is a five minute edit and a test run. This is the harness the article is on about, in its smallest form. Even if you never move off your current provider, it's worth doing, because prices and models change every few months.
3. Build a small test set before you swap anything
You can't tell whether a cheaper model is "at parity" by trying it twice and squinting. Collect a few dozen real examples for each routine step, with the answer you'd expect, and run them against the current model and the candidate. Watch three things: whether the answers are right, how long they are, and how often the call has to be retried. Those last two are exactly where Gartner says the saving shrinks, so measure them on your own data rather than taking anyone's word for it.
4. Decide where the open model runs
This is where the control point gets real. You've broadly got two options.
Rent it from a host. Lots of providers run open-weight models behind an API. It's quick and cheap, but your data still leaves the building. You do get to choose the provider and often the region, which is more choice than you get with a single closed model.
Run it yourself. On your own server or hardware, the data stays in-house. The catch is that you now own a server with a graphics card in it, and that server needs patching, monitoring and keeping off the open internet like any other. I've written up what a local AI server seems to take on my own site, including how much graphics memory you need and why. If the data question is what's bothering you, my post on pasting client details into ChatGPT covers the account and contract side.
Plenty of businesses will end up with a mix: a frontier model for the few hard steps, an open model for the routine ones.
5. Treat the model's output like any other input
A cheaper model making tool calls is still something making tool calls inside your system. Validate what comes back. Check it's only allowed to touch what it should. If a routine step says "update this customer record", the code should check that it's a real customer, the right one, and that the change makes sense, whichever model asked. A good harness does that regardless, and it's a lot easier to add before you start swapping models than after.
Don't rush it
None of this needs doing this week. Your app works, the 90% figure is a 2030 assumption, and moving a step to a different model without testing it is a good way to break something that was fine. But if your AI bill is growing, or you're uneasy about where your data goes, the first two steps cost very little and make everything after them easier. We wrote about the cost side last month in AI models got cheaper, your app might not have noticed.
If you've got an app with AI built in and you're not sure how it's put together, our code audit (£250 + VAT) will tell you, in plain English, where the model calls are, what they cost you and what we'd change, with a fixed price for each fix. If it was built with an AI tool in the first place, have a look at our page on taking on AI-built apps. And if you'd rather someone just kept an eye on it, including the server if you go down the self-hosted route, Look after it starts from £355 a month + VAT.
Cheers, Rob
