Harvey is burning through a lot more AI tokens. Trillions more.
The legal AI startup used 1 trillion tokens in January, CEO Winston Weinberg told the “Sourcery with Molly O’Shea” podcast, released on Tuesday. Harvey’s AI product is used by law firms and corporate legal teams to help draft, review, and analyze documents.
The startup’s monthly token use has since ballooned roughly 12X.
Weinberg said the company was on pace to use 12 trillion to 13 trillion tokens in a month. A Harvey spokesperson confirmed to Business Insider that the estimate referred to May.
“The usage is getting insane,” Weinberg said.
Tokens are the chunks of text that AI models process when users type prompts and receive answers. They have also become a key measurement of AI use and billing. Generally speaking, the more complex an AI task is or powerful the AI model processing the request, the more tokens are burned.
The remarks from Harvey’s CEO come as several tech companies are paying closer attention to their AI usage costs. Business Insider recently reported that some startups are pulling back from “tokenmaxxing” after receiving high AI bills.
A few examples of the token reassessment: Coinbase is routing some prompts to cheaper models, and Uber executives have questioned whether spending on AI coding tools is translating into enough measurable product gains.
Weinberg said the next challenge for companies is deciding the right level of AI use for each job.
“You don’t want frontier intelligence running every task. It’s too expensive,” Weinberg said in a statement to Business Insider. For example, “a change of control review warrants it. A first-pass document summary doesn’t,” he wrote.
On the podcast, Weinberg compared the coming debate over AI spending to how law firms justify their bills to clients. He called it the “billable hours problem.”
“When you get a bill from a law firm, it says in six-minute increments what people did, and then the hourly rate,” Weinberg said. “Why did they do that? It’s because they’re trying to show ROI.”
As companies spend more on tokens, he said, they will face growing pressure to explain what that spending actually accomplishes.
“This is kind of like the main problem that I think the entire world is about to hit,” he said, “Which is, ‘I just spent $1 billion on tokens. Where’s my ROI?'”
