Mark Zuckerberg shocked investors last week with the eye-watering amount that Meta Platforms is spending on developing artificial intelligence. The company’s share price has fallen by almost 12 per cent since its chairman and chief executive forecast that its capital expenditure would be between $35 billion and $40 billion this year, which would be up to 42 per cent higher than the year before.

It is not alone. All the Big Tech companies are increasing their outlay on AI. In the latest quarter, Microsoft spent $14 billion, a 79 per cent rise and a number that it expects to “increase materially” in the year ahead. Alphabet spent $12 billion in the last quarter, up 91 per cent year-on-year.

Looking at the data, energy and computing power required for generative AI, it’s not surprising that training these ever more advanced models, which can do ever cleverer things, is becoming ever more expensive. Yet while Big Tech shells out billions of dollars, smaller businesses are finding that the cost of using generative AI models is going down. So much so, it’s opening up new frontiers for companies.

Take Olio. The popular platform to give away unwanted food items explored using generative AI in October 2023, when it found that it would cost an estimated $80,000 for a year’s use. In January 2024, the business enquired again and found it would cost $35,000. Later it fell to $4,000. Many others, in different sectors, tell a similar story. One legal technology firm said its cost had fallen by 80 per cent since October.

As Joshua Wohle, the chief executive of Mindstone AI, put it: “The cost of these models just keeps falling through the floor.”

Sean Williams, who runs autogenAI, has seen the costs of using the underlying “large-language models” fall by 50 per cent in the past year. He compared these foundation models to “electricity”, saying they had become “essentially commoditised”.

So what is going on? It’s difficult to set out a one-size-fits-all when it comes to cost, as some businesses have bespoke deals. However, it seems that increasing competition between OpenAI, Anthropic, Google, Mistral AI and open-source models such as the Technology Innovation Institute’s Falcon and Meta’s LLaMA means that all are trying to tempt customers. There is simply more choice available.

OpenAI said it was not dropping its prices, but take a look at its newer models and they are much cheaper. For those of us who aren’t chief technical officers, here’s a quick crib sheet. Developers pay to integrate application programming interfaces from OpenAI into their own applications, to access certain versions of GPT. Costs typically are based on the number of “tokens” that are processed, the information that goes in and comes out. Output costs more than input. One token is about four letters; for instance, it would buy you “word” or “help”.

Consider OpenAI’s latest model, GPT-4 Turbo. It is far more powerful than its predecessor GPT-4. Its context windows are larger, for one thing. This is the amount of information that the models can process at one time. This also cuts the work that developers have to do. “You can just push more data over to OpenAI or whichever other platform to do the work for you,” one chief technical officer remarked.

For this latest version of GPT, with a context window of 128,000, fresher knowledge and the broadest set of capabilities, it costs $10 for a million input tokens and $30 for a million output tokens. For GPT-4, the older model with an 8,000 context window, it is three times the cost for input, $30 for a million tokens, and twice the price, $60 for a million output tokens; while a 32,000 context window is $60 for a million input tokens and $120 for a million output tokens.

Some new models are being offered at reduced rates, for example Google’s Gemini 1.5Pro is temporarily free for the testing of a million context window.

As one company’s lead AI manager described it: “It’s a race to the bottom, pricing-wise. Unless someone establishes market dominance with amazing new features or a huge improvement in the output, it’ll hopefully stay that way. At the moment, it’s very good for consumers”.

Katie Prescott is Business Technology Editor of The Times. [email protected]