Turning on an AI tool takes about four seconds: click a toggle, approve a permission screen, and the tool starts working. No ticket or meeting needed. And that speed is the whole appeal, and it’s also the problem.
Most companies now have something written down about AI, like a policy or a list of approved tools. But the document in your shared directory and the tools running inside your systems this morning may describe two different companies. One is aspirational, and the other has access to your data.
That gap is where the risk lives. Not in the tools, but in the questions nobody asked before the toggle flipped.
Summary
The answers come down to permissions, interconnects, and containment: split read access from action access before approving anything, review how your approved tools combine rather than evaluating them one by one, and sandbox AI so it only reaches what you explicitly allow. Slingshot’s Steve Anderson and Doug Compton walk through the scoring rubric that turns tool approval from a months-long debate into an hours-long decision, plus the shutoff plan that keeps a failed tool from becoming an outage. The companies getting this right are not the ones with the longest policy document.
Start With the Data, Not the Demo
Almost every AI evaluation starts in the wrong place: ‘what can it do?’ Someone watches a demo and gets excited about the output, features, or possibilities. The better first question is far less interesting but far more useful.
“One of the first things I look for is what data it’s accessing,” said Steve Anderson, Principal Developer and AWS Architect at Slingshot.
Doug Compton, Slingshot’s Director of AI Engineering, framed it as two questions. “The data is the big one, and then what can it do with that data?”
Those are separate risks. Reading your customer records is one thing. Sending an email on your behalf is another. Most tools let you split those permissions apart, and most companies never look closely enough to do it.
“With Gmail, you can tell AI to be read-only, or you can allow it to draft an email for you,” Doug said. “You can give the AI actions it’s allowed to take. But a lot of the time, those permissions just don’t go far enough.”
The scope also tends to be wider than people expect. “At Slingshot, when we looked at hooking up Google Drive to the OAuth application, the permission set was shockingly broad,” Steve said.
So when your team brings you a tool, ask to see the permission screen rather than the vendor’s feature page. Those two documents rarely tell the same story.
Safe Tools Can Add Up to an Unsafe System
Here is the part that surprises most leaders: you can approve every tool individually, correctly, and still end up exposed.
“One connector you enable might give AI access to sensitive data, and then another tool you enable gives it the ability to do something with that data, possibly making it public,” Doug said. “So you have to look at the interconnect between these tools as well.”
Tool A reads. Tool B publishes. Neither one is dangerous alone, but together they’re a leak with a schedule, and reviewing tools one at a time misses it entirely.
The same thing happens over time. A tool you approved in March isn’t the same tool in October, and new capabilities arrive enabled by default. “We’ve seen AI tools push updates that were opted in by default,” Steve said.
His fix is unglamorous yet effective. “Whenever there’s a release, you have to check what the release notes say before you install that update. And you should have a policy that dictates the cadence you’re reviewing your vendors.”
Annual vendor review made sense when software changed annually. AI tooling doesn’t run on that calendar.
“Nobody” Is the Most Expensive Answer
Ask who owns your AI tools. If the honest answer is that no single person does, you’ve found your most urgent gap.
“That’s the gap a lot of IT teams are dealing with right now: being able to manage AI tools and keep them locked to policies,” Steve said. “Your IT team should be the one ensuring whatever tools are in place are approved and operating within your policies.”
Central control helps, but it has a ceiling. Doug walked through where it stops: “Admins can enable and disable tools from a centralized location. But that’s really just restricting the desktop app. Somebody can still install a tool locally, or reach one across the internet. Disabling a tool in the admin console doesn’t mean the AI can’t still get to sensitive information.”
That is shadow AI, and it’s not hypothetical. “Shadow AI is a very difficult thing that people are struggling with right now,” Steve said. You cannot monitor your way out of it, either. “Without putting spyware on everything, you have to trust your people to an extent to follow the policies you put in place.”
So your policy only works if your people understand it well enough to follow it, which is a training problem long before it’s a technology problem.
Decide What Sits Between the Tool and the Outside World
One question separates a contained mistake from a public one. When this tool produces something, who sees it before your customer does?
Both Doug and Steve have watched what happens when the answer is nobody. “The worst thing I’ve had AI do is automatically update AWS in our dev environment,” Doug said. “I was watching it, so it wasn’t a big deal. But it raised my eyebrows at what it’s capable of.”
Steve’s example was quieter and more instructive, because nothing broke. “I enabled AI to create Jira tickets based on reviews and end-to-end testing automatically. Nothing awful, but the volume of tickets and the nitpicky level became a flood of information. It made the project difficult to navigate.”
The system did exactly what it was told. It just buried a team in noise, which is its own kind of failure.
Better instructions help, but they don’t guarantee much. “A good prompt around the tools alleviates a lot of that, but it’s trial and error,” Doug said. “You don’t know you need that section of the input until the tool does something stupid or dangerous. And the prompt still doesn’t stop it from doing it again; it just makes it harder.”
And your non-technical teams can’t catch what they were never taught to look for. “This is so new and so different from what we’ve had to deal with in the past that companies really need to give their employees training just to know what’s possible and what to look out for,” Doug said. “Some of this isn’t obvious.”
Contain It Before You Have to Kill It
Most companies never consider a containment layer, and it does more work than another paragraph of policy ever will.
“You need to contain AI in a sandbox, so it only has access to the things you allow,” Doug said. “It doesn’t need access to your whole computer, your documents, or the network unless you explicitly say so. Sandboxes are almost like virtual machines. You control exactly what files it has access to, and you can put a firewall between it and the internet.”
Containment also buys you an exit, since it’s more than likely that a tool will one day change, break, or fail a review. “Hopefully you had a shutoff plan to begin with,” Steve said. “If you’ve built your entire application around the AI, it’s a single point of failure. If you have to shut things down while you fix it, you’re just going to be down.”
Decide the borders while the tool is still optional, and the day you have to pull it stays a decision, not an outage.
A List of Approved Tools Is Not Governance
Most AI policies stop at a list: use only the tools leadership signed off on. The list is a fine start and a poor finish, for two opposite reasons.
It goes stale faster than anything else in your stack. “That list of tools is moving and changing quickly, much more than other types of software,” Doug said. “If you want to get the most out of AI, you need to look at that approved list more frequently. You’d hate to miss out on something that would help the company in a significant way.”
A list also tells your team what was decided without telling them how to decide. Steve’s answer is a guide. “You want a scoring rubric for whether or not you should approve a tool. Think of things you should look at when evaluating: what are the things that are okay for a tool to do, and what are our hard lines where we won’t approve it without particular remediations in place?” Write the rubric down once, and every future tool gets evaluated in hours instead of months.
Skipping that work costs more than it looks. “There are a million stories of people who deleted production databases because they weren’t careful,” Steve said. Doug pointed at the quieter version. “Your company could send out a bunch of emails written by AI that were wrong or embarrassing, and now the world knows you did stupid stuff.”
Those are mistakes you make. Your intellectual property is exposure you may never see. When OpenAI announced in September 2026 that its agents had cracked part of the Navier-Stokes problem, mathematicians working on the adjacent problem publicly questioned whether their research data, run through OpenAI’s Codex tool, had fed the effort. OpenAI denied accessing their work and said no specific user data was involved, while recognizing it couldn’t fully rule out an indirect connection.
Scale is what makes that dangerous for a company your size. “If you don’t have a big enough data population, you can’t de-identify things,” Steve said. If your company is the only one in the world doing something a particular way, stripping your name off it protects almost nothing. He landed the whole question in one comparison. “Just like you wouldn’t give your intern the keys to the kingdom, you shouldn’t give them to AI either.”
The Toggle Is the Easy Part
None of this argues for slowing down; it argues for knowing what you just agreed to.
The companies getting AI right are not the ones with the longest policy document. They are the ones who can answer a short list of questions about every tool they run: what data it touches, what it can do with that data, what other tools it can reach, who owns it, who checks its work, and how you turn it off on a bad Tuesday. That is governance. The policy is just where you write it down.
Your team can turn on an AI tool in four seconds. The only real question is whether anyone can tell you what happened in the next four.
Where Policy Meets Training
Written by: Savannah Cherry
Savannah leads marketing and new business at Slingshot. She writes, posts, and creates all things Slingshot, and helps companies navigate working with a tech partner for the first time. While she isn’t developing software, her CIS minor and a tendency to tinker with AI tools to streamline her own work keep her up to speed on the team’s work. She co-organizes and hosts the Louisville AI Exchange, and she can’t rest until all her work is done.
Expert: Doug Compton
Born and raised in Louisville, Doug’s interest in technology started at 11 when he began writing computer games. What began as a hobby turned into his career. With broad interests that range anywhere from snorkeling, science, WWII history and real estate, Doug uses his “down time“ to create new technologies for mobile and web applications.
Expert: Steve Anderson
Steve is one of our AWS certified solutions architects. Whether it’s coding, testing, deployment, support, infrastructure, or server set-up, he’s always thinking about the cloud as he builds. Steve is extremely adaptable, and can pick up the project and run with it. He’s flexible and able to fill in where needed. In his spare time, he enjoys family time, the outdoors and reading.
Frequently Asked Questions
Six questions cover most of the risk: what data the tool can access, what actions it can take with that data, which other tools it can reach, who owns it internally, who reviews its output before a customer sees it, and how you shut it off. Start with the permission screen rather than the vendor's feature page, because the two rarely describe the same tool.
Shadow AI is any AI tool employees use without IT approval or visibility. Admin consoles only restrict the desktop apps you control, so someone can still install a tool locally or reach one across the internet. Managing it is a training problem before it is a technology problem: your policy only works if your people understand it well enough to follow it, and short of putting spyware on every machine, you have to trust them to.
Yes. One connector may give AI access to sensitive data while another gives it the ability to publish or send. Neither is dangerous alone, but together they create a path for that data to leave your company. Reviewing tools one at a time misses this entirely, so evaluate the interconnect between tools, not just each tool in isolation.
A sandbox is a contained environment, similar to a virtual machine, where you control exactly which files an AI tool can reach and whether it can access the internet. It does not need your whole computer, your documents, or your network unless you explicitly allow it. Containment also gives you an exit: if the tool changes, breaks, or fails a review, you can pull it without taking the business down with it.
No. A list goes stale faster than anything else in your stack, and it tells your team what was decided without telling them how to decide. Pair it with a scoring rubric that defines what a tool is allowed to do and which hard lines require remediation before approval. Write the rubric once and future tools get evaluated in hours instead of months.
Far more often than the annual vendor review most companies run. AI tools push updates that arrive opted in by default, so a tool you approved in March is not the same tool in October. Check release notes before installing an update, and set a policy that dictates the cadence for reviewing vendors.
