4 Vendor Moves That Can Break Your AI Product Overnight

You can write clean code, run a tight sprint, and ship an AI feature your users love. Then one morning it slows down, costs three times as much, or just stops. Nothing on your side changed.

That’s the strange new math of building with AI. The model doing the hardest work in your product belongs to a company you don’t control. It lives in someone else’s account, sits on someone else’s roadmap, and carries a price set by someone else’s finance team. They can move it whenever they want, and they don’t need your approval.

Most technology leaders already accept that deal with their cloud provider. They plan for outages and price shifts because they’ve survived a few. Model providers get filed somewhere else entirely, closer to a software purchase than infrastructure. That filing error is where products break.

Here are the four moves your AI vendor can make, and what to have ready before they do. 

They Retire the Model You Built On

Deprecation is the move that lands first, and it arrives faster than most enterprise teams expect. Steve Anderson, Principal Developer & AWS Architect at Slingshot, watched it hit a client project directly. 

“We had a client on a specific model in Bedrock, and AWS sunsets those a lot faster than other services,” Steve said. “We’re used to long-term releases with two-year support periods. Model lifespans run significantly shorter than that. We’re talking months in some cases.”

Months, not years. Miss the notice, and you get an outage in a feature your customers depend on. Whole tool categories shift the same way. Chris Howard, Slingshot’s CIO, pointed to prototyping tools like Lovable and Bolt that the team dropped once those capabilities folded into platforms they were already using. 

Catching that movement is harder than catching a broken API. “In the past, if an API got deprecated and replaced, you had a series of tests to run to see what broke,” Chris said. “With these models, it’s harder.” Traditional tests answer a binary question. Models fail on a gradient instead, quietly returning weaker output in ways your assertions never flag.

Steve’s answer to that gap is short and specific: evals. Think of them as regression tests for probabilistic systems. You define what good output looks like for your actual use case, score it, and rerun that scoring every time anything underneath you shifts. Without evals, a model swap becomes a guess. With them, it becomes a measurement, which is the only thing that makes the next move survivable.

They Change the Model and Keep the Name

The scarier version of deprecation is the one that never announces itself. Some tools quietly fall back to a cheaper, smaller model once usage caps hit. Same product, same invoice, weaker answers. Your monitoring stays green the whole time.

Whether that can happen to you comes down to a decision made months earlier in the architecture. “Because we’re using Bedrock, we’re not saying ‘use Anthropic,'” Steve said. “We’re naming the specific model. And because we’re paying for that provisioning, there’s no downgrading.”

Naming a vendor leaves routing decisions to the vendor. Naming a model and paying for dedicated capacity moves those decisions back to you, with the cloud provider handling hardware and management while the model runs inside your account. “Nobody’s moving the goalposts on you,” Steve said.

Not every complaint about degraded AI comes from the vendor. Chris described a client comparing Slingshot’s document generation system to a one-off prompt in a chat window. The chat window finished faster, so the client assumed something broke. Nothing had. The production system carries guardrails, validation, and scale that a single prompt never touches. Before you go hunting for a vendor change, make sure you’re comparing the same thing.

They Change the Economics Mid-Flight

Pricing is the move most people brace for and the one that has caused the least damage so far. Free tiers grow annual fees. Usage tiers appear. But the math usually breaks for a different reason.

“If you build a custom solution around something that’s free right now, that’s risky,” Chris said. “You have to identify that as a risk early and be prepared for it.” Steve pushes it further: not having a backup plan is a risk itself. The dependency isn’t the problem. The single dependency is.

“You can’t just look at cost per token,” Steve said. “Some models solve the same problem with fewer tokens, but those tokens are more expensive. You’ve got to look at cost to solve.” That changes model selection entirely. Slingshot has shipped products on Haiku because a smaller model handles the job well and beats throwing a sledgehammer at it. Right-sizing upfront builds a cushion that survives a later price increase.

Here’s the twist: the biggest cost surprises usually aren’t the vendor’s doing. Token prices have stayed fairly predictable. Your users haven’t. “You missed some behavior. People use the product differently than you expected. Or you just didn’t plan for it,” Steve said. Chris’s fix is telemetry. Instrument token usage at the feature level and alert on it the way you’d alert on compute.

They Change the Terms and Shrink What You Can Build

The last move doesn’t touch your product’s performance at all. It changes what you can build, and where your data travels while you build it.

“We’ve had to cancel tools we really liked because we read their data privacy documentation and it just isn’t good,” Chris said. He described transcription tools reserving the right to train on customer conversations even under a paid subscription. You only find that out by reading the terms, and the terms are miserable to read. Steve thinks that’s deliberate, not accidental.

Chris found the obvious workaround: let AI do the reading. Pull a vendor’s terms of use and privacy policy into a PDF, hand it to a model, and ask it to surface the risks. Run the same pass across three or four competitors and compare what comes back. It turns an afternoon of legalese into a decision you can defend.

The harder question sits underneath the document. “You’ve got to know where the data’s going,” Chris said. “That’s getting harder and harder to determine.” Tracing that route means going past your direct vendor to their vendors, because your data doesn’t stop at the tool you signed for.

That’s a large part of why Slingshot keeps landing on Bedrock. Treating architecture as the answer to a legal question sounds odd until you realize a closed system leaves far less to argue about.

And when the notice arrives, you’ll typically get about 30 days. Chris called that window challenging. Steve was more direct: you don’t want a deadline that forces you to write your evals and run them at the same time. The window isn’t for building your defense. It’s for using it.

Four Moves, One Defense

Retire the model. Swap it without renaming it. Reprice it. Rewrite the rules around it. Different moves, same category of risk: a dependency you don’t control.

Which is why the preparation looks identical every time. Evals written and running before anyone needs them. A second model already priced, tested, and reachable. Token usage watched at the feature level. Architecture that names exactly what you’re running and where your data travels. Terms understood before you sign, not after something changes.

None of that stops a vendor from moving. It just means the change costs you a sprint instead of a product.

So the question isn’t whether your vendor makes one of these moves. It’s whether you’ll hear about it from your evals or from your customers.

Slingshot