Hackers have found another way to make money from AI: steal access to someone else's account, sell it to strangers, and leave the account owner to pay the bill.
It's called LLMjacking, and Fireworks AI says its recent security breach has links to criminals involved in the scheme.
On November 9, Fireworks confirmed that attackers had accessed customer information, including billing records and, in some cases, datasets and parts of unfinished AI models.
The worrying part? Companies could end up paying for AI services they never used, while sensitive information they've stored with an outside provider could also be exposed.
And Google's recent research suggests criminals have figured out how to turn stolen AI access into a profitable business.
Fireworks AI's breach turned out to be worse than first reported
Fireworks AI helps companies run AI models and customize them with their own data. Businesses use its services so they don't have to maintain all the expensive computing equipment themselves.
On October 6, Fireworks discovered that someone had used its internal credentials to access information associated with a limited number of customer accounts.
The attackers got hold of environment variables used in AI evaluation jobs. These can contain API keys and tokens, which are essentially digital passes that allow applications to connect to other services. In its security notice, Fireworks said it hadn't found evidence that customer models or datasets had been accessed.
Unfortunately, the investigation didn't end there.
On November 9, Fireworks revealed that attackers had also accessed customer billing addresses, payment histories, and credit usage. For some customers, the exposed information included datasets and portions of unfinished AI models.
Those model files could be particularly concerning for businesses developing their own AI technology. They contain numerical parameters that determine how an AI model behaves, and companies may have spent considerable time and money creating them.
Fireworks says payment card numbers weren't affected, and it hasn't reported that complete customer models were stolen.
The company also discovered another intrusion connected to a third-party cloud provider.
After the provider experienced a security incident on October 23, attackers reached Fireworks' development machines the following day. They then ran unauthorized software that resulted in additional customer data exposure.
Fireworks detected the activity on October 24 and says it hasn't found further successful unauthorized access since October 25. The investigation is still ongoing.
So, how are hackers making money from stolen AI accounts?
Here's where the story gets interesting.
When a company uses an AI model through an API, it typically pays based on how much it uses the service. Every request consumes computing resources, which can add to the bill.
Now, suppose a hacker gets hold of the company's API key. Depending on the permissions, the hacker could use the company's account to send AI requests. The provider sees activity from a valid credential and may charge the legitimate account owner.
But some criminals have found a way to make even more money: they resell the stolen access to other people.
According to a November 2 investigation by Google Threat Intelligence Group and Mandiant, criminals use underground services called LLM Mixers to sell discounted AI access.
Think of these services as unauthorized middlemen. They combine access from stolen accounts and let paying customers use expensive AI models for less.
The criminals collect the payments, while the companies whose credentials were stolen may end up covering the usage costs.
And we're talking about some seriously expensive AI bills.
Google investigated cases where fraudulent AI usage exceeded $100,000 in a single day. Some organizations accumulated charges reaching $1 million before the stolen access was revoked.
In another case, hackers made more than 500,000 AI requests in less than eight hours, which generated nearly $1 million in token costs. To be clear, those figures come from Google's investigations of other incidents. Fireworks hasn't disclosed similar financial losses from its breach.
However, Fireworks says the evidence points to a financially motivated cybercriminal group associated with LLMjacking.
The stolen AI access isn't the only thing companies should worry about
While unexpected AI bills are one concern, Fireworks' breach also raises questions about customer information stored with outside AI providers.
Say your company is developing a custom AI model. You might upload proprietary datasets to a service like Fireworks so you can train or customize the model without buying expensive hardware.
You'd naturally expect those files to remain accessible only to authorized users.
But if attackers gain access to the provider's systems, your company's information could be caught up in the breach, even if nobody at your business made a security mistake.
That's what makes Fireworks' latest disclosure concerning.
The company confirmed that attackers accessed certain customer datasets and portions of incomplete model weights. It hasn't said that complete models were stolen, and the consequences will depend on the information contained in the affected files.
There's also the question of how much control companies have over the security of their AI providers.
Remember the second Fireworks intrusion? It began with a security incident at another cloud provider. Attackers then reached Fireworks' development systems, where they accessed additional customer information.
So, a security problem at one provider ended up affecting customers of another.
Some customers have already decided to move their business elsewhere.
Forbes reported that two customers had moved away from Fireworks following the incidents. One also criticized how the company communicated about the breaches.
Fireworks says it hasn't detected an impact on its Inference, Training SDK, or Nexus services, which use zero data retention by default.
That means those services generally don't keep customer data after processing it. However, the company did confirm exposure involving information stored elsewhere in its systems.
For businesses, it's worth asking their AI providers exactly which information they store, who has access to it, and how it's protected.
What should businesses do after a breach like this?
Fireworks says it has revoked compromised credentials and restricted access to affected systems. The company also brought in Palo Alto Networks' Unit 42 cybersecurity team to help investigate.
It has temporarily disabled self-service dataset uploads and downloads for certain services while the security work continues. Affected customers have been advised to replace exposed credentials and check their accounts for suspicious activity.
If your company uses other AI providers, Google's research offers a few lessons, too.
First, limit which services your API keys can access. If an application only needs one AI model, there's little reason to give its credentials access to every service on your account. It also helps to watch for unusual spikes in usage.
For example, if your application normally makes a few thousand AI requests a day, suddenly seeing hundreds of thousands should prompt an investigation. Setting spending limits, where supported, can also help prevent unauthorized AI usage from turning into an enormous bill.
And if you're uploading proprietary data to an AI provider, it's worth checking how long those files are stored and which employees or systems can access them.
Fireworks is still investigating the full extent of the breach, so affected customers may learn more as the company releases updates.
One finding from Google's LLMjacking research offers an interesting look at the criminals' business.
Researchers spotted outages at an underground AI service that they suspected happened when compromised credentials were revoked.
In other words, once the account owners cut off access, the criminals' service stopped working.
For companies, there's a financial reason to catch suspicious AI usage early. Every hour that stolen credentials remain active gives criminals more time to use paid AI resources and sell access to someone else's account.
