Nvidia’s revenue climbed to $44.1 billion last quarter, yet one of the chip giant’s most important metrics doesn’t have a dollar sign in front of it.
“OpenAI, Microsoft, and Google are seeing a step function leap in token generation,” said Nvidia CFO Colette Kress on the company’s Wednesday earnings call.
“Microsoft processed over 100 trillion tokens in Q1, a fivefold increase on a year-over-year basis,” she continued.
Throughout May, top tech executives — many of whom are also some of Nvidia’s largest customers — have been boasting about their token growth.
Though somewhat hard to track from outside the AI cloud or foundation model companies themselves, tokens are the base unit to measure AI inputs and outputs.
They include pixels, word segments, or audio. But no matter the content, all AI breaks down into tokens. As AI tools mature, the number of tokens generated for AI outputs, or inference, is growing faster than many expected.
When Google CEO Sundar Pichai said monthly tokens produced across Google’s products had increased by a factor of 50 in the last year, the Google I/O audience gasped.
“Explosive token growth is what really matters, in the longer term,” wrote Morgan Stanley analysts ahead of Nvidia’s latest earnings call, after which Nvidia’s share price climbed to $134.
Why tokens matter
Forrester analyst Alvin Nguyen explained that it’s not a perfect metric, since tokens can vary in size based on the form of content they represent.
“It isn’t a clear way to make apples-to-apples comparisons, but it is the closest to a standard that we have without needing more data and analytics,” Nguyen said.
Nvidia CEO Jensen Huang sees the rise of the “token” in the conversations of top tech executives as a sign that AI tools are providing value.
“Where companies are starting to talk about how many tokens they produced last quarter and how many tokens they produced last month. Very soon we’ll be talking about how many tokens we produce every hour, just as every single factory does,” Huang said at Computex in Taiwan last week.
With its dominant market share, Nvidia directly benefits from almost all token growth. Some analysts say the demand is growing faster than the current data center stock can handle.
“Every hyperscaler has reported unanticipated strong token growth,” the Morgan Stanley analysts wrote, adding that plentiful anecdotal evidence of more inference demand than existing infrastructure can support strengthened their conviction.
The problem with tokens is that there’s no easy way to gauge their growth unless companies release numbers. But it’s safe to say that when executives share any indication of demand for tokens, investors will be paying close attention for the foreseeable future.
