IBM CEO Arvind Krishna used his company’s Q3 earnings call this month to make a statement that would have sounded wrong two years ago: open weight AI models are now good enough and cheap enough that running them on your own hardware is the obvious choice for most enterprise work. His framing was blunt. Open weight models deliver roughly 90 percent of frontier model performance at 10 to 25 percent of the token cost.
“It’s a simple question for any enterprise,” Krishna said. “Would you like something that’s 10 percent of the cost if it’s 90 percent as good as the best, or do you want to pay that 10 times higher cost?” For Indian companies weighing AI budgets line by line, that question lands hard. The same maths shows up in every enterprise AI discussion now: if routine work can run on hardware you already control, expensive API calls can be reserved for the tasks that genuinely need them. Here is what he actually said, why the economics shifted, and what to check before moving AI workloads on-prem.
What IBM’s CEO Actually Said
Krishna walked investors through how quickly the gap between closed frontier models and open alternatives has narrowed. In 2024, he said, the difference was not close. “If you ran some assessment, some benchmark, and it was broad-based, not narrow, the top model might score a 70, and the open model might score a 20 or a 30.”
By late 2025 the gap had shrunk to something like 70 against 50. Now, in his telling, open weight models run at “anywhere from 25 percent to 10 percent of the token cost of the frontier models” while being “90 percent as effective.”
The follow-up matters more than the benchmark talk. Krishna argued that most daily enterprise tasks do not need the best model in the world: “summarizing an email, examining a contract, answering a customer call, composing an email, all of those can be done as well with the much cheaper answer.” Some workloads genuinely require frontier performance. The vast majority, in his view, do not pay for it.
On-Prem AI Is Back on the Table
The cost argument points straight at infrastructure. If the model is cheap enough to run yourself, the question becomes where it runs. Krishna said IBM is seeing booming interest in on-prem AI infrastructure, and that most businesses can build a useful AI setup in just two or three racks.
Two or three racks is a fraction of what people imagine when they hear “AI data centre”, and that reframe is deliberate. The hardware barrier has dropped even as the software barrier has become the real problem.
The Hard Part Is Not the Hardware
Krishna was candid about where enterprises get stuck: talent and systems work, not silicon. “The question that the enterprise struggles with is: where are my kernel engineers? Where are the people who worry about the underlying network?” he asked. “Who can make sure that the model is clustered on GPUs that are close to each other in network proximity? Who can make sure that you’re doing the right caching of results so you get an incredible response?”
His answer to that gap is more IBM software to ease deployments, plus expectations that on-prem AI demand will pull sales of the company’s Spyre accelerator cards for mainframes. The pieces fit: the same enterprises that trust mainframes with core banking now have a reason to attach inference hardware to them.
The earnings numbers around that strategy were mixed but instructive. Current-generation z17 mainframes have delivered $1 billion more revenue than any previous model, and the newer rackable versions produced what IBM called the strongest midrange launch in IBM Z in nearly two decades. Mainframe revenue overall still fell 20 percent, which IBM expected 18 months into the z17 cycle. Infrastructure revenue grew 5 percent year over year to $3.7 billion, slower than Dell, HPE and Lenovo managed in the same quarter. Software did better at 9 percent growth to $7.8 billion, with Red Hat up 11 percent and annual recurring revenue up 30 percent.
What This Means for Indian Enterprises
The arithmetic Krishna describes is especially relevant in India, where per-token pricing at frontier rates multiplies fast across millions of customer interactions. A bank answering routine service queries, an IT firm summarising contracts at scale, or an e-commerce team writing product copy does not need the most expensive model for every call.
Running open weight models on-prem also keeps data inside your network. For sectors with strict data locality expectations, from BFSI to government services, that is often worth more than the cost saving. The two benefits stack: cheaper tokens and no data leaving the building.
What to Check Before You Move Workloads On-Prem
The pitch is real, but the gap Krishna described is also real. Use this checklist before committing:
- Benchmark on your tasks, not public leaderboards. The “90 percent as good” figure is an average across broad assessments. Summarisation and drafting may be closer to 95 percent; complex reasoning can fall further behind. Test on your own documents.
- Price the whole stack. Token cost is only one line. Add GPU or accelerator cards, power, cooling, networking, and the engineers to run them. Two or three racks is affordable; the team to operate them is the scarce part.
- Plan caching and clustering from day one. Krishna’s own list of enterprise struggles is mostly systems work: GPU proximity, network design and result caching decide whether responses feel instant.
- Keep a frontier fallback. Route the small share of tasks that genuinely need top-tier models to an API, and run everything else locally. That hybrid is exactly the shape his argument implies.
- Watch the open weight releases. The gap narrowed from 70 versus 25 to 70 versus 50 to today’s pricing. Models change fast enough that an evaluation from six months ago is already stale.
Open Weight Is Not the Same as Open Source
One clarification saves procurement teams a lot of confusion. “Open weight” means the trained model weights are published and anyone can run them. It does not always mean the training data, training code or full recipe are public, which is what “open source” would imply. Some open weight releases come with generous licences, others restrict commercial use or large-scale deployment.
The practical takeaway stays the same either way. You can host the model yourself, keep data inside your network, and pay hardware costs instead of per-token API bills. Just read the licence before you build a product on it, and confirm the model card covers the languages and scripts your workload needs, which matters especially for Indian language support.
FAQ
What are open weight models?
Open weight models are AI models whose trained weights are published, so anyone can download and run them on their own hardware. Unlike fully closed frontier models, they can be hosted on-prem, fine-tuned privately and served without paying per-token API prices.
How much cheaper are open weight models to run?
IBM CEO Arvind Krishna puts current open weight token costs at 10 to 25 percent of frontier model pricing, while delivering about 90 percent of the performance on broad assessments. The exact saving depends on your hardware costs and the tasks you run.
Can enterprises really run AI models on their own servers?
Yes. Krishna says most businesses can build useful AI infrastructure in two or three racks. The harder part is operational talent: kernel engineers, network design for GPU clustering, and caching for fast responses. IBM is building software to ease that deployment work.
Which is better for business, open weight or frontier models?
For most routine work, such as summarising email, reviewing contracts or answering customer calls, open weight models on-prem are the better value. Keep frontier APIs for the smaller share of tasks that need top-tier reasoning, and evaluate both on your own data before deciding.
