How to Run an AI Tool Discovery Audit in Your Own Organization

How to Run an AI Tool Discovery Audit in Your Own Organization

A compliance officer at a mid-size financial services firm recently told me that during a routine vendor review, they discovered 47 distinct AI tools being used across the organization. They had approved three. The other 44 ranged from benign (a grammar checker) to genuinely alarming (a free-tier summarization tool where an analyst had been pasting client portfolio data). Nobody was acting maliciously. People were just trying to work faster.

This is the shadow AI problem, and it is now a core information governance issue. If you do not know what AI tools your employees are using, you cannot assess where your data is going, what third parties are training on it, or whether you are in compliance with obligations under HIPAA, GLBA, CMMC, FERPA, or your own contractual NDAs. Running an AI tool discovery audit is not optional anymore. Here is a practical framework for doing it.

Start with Network Telemetry

Your network already knows more than you do. DNS logs, proxy logs, and firewall data contain a record of every external service your employees connect to. The challenge is filtering signal from noise.

Start by building a watchlist of known AI service domains. This includes the obvious ones (openai.com, anthropic.com, gemini.google.com, midjourney.com) but also the long tail: perplexity.ai, jasper.ai, copy.ai, runway.ml, otter.ai, fireflies.ai, and dozens of others. Services like Hugging Face (huggingface.co) are worth flagging too, since employees with even moderate technical skills can access open-source models through hosted inference endpoints.

Most SIEM platforms and next-gen firewalls can be configured to alert on or log connections to a custom domain list. If you are running CrowdStrike Falcon, Palo Alto Cortex, or Zscaler, you likely already have the infrastructure. The key is creating and maintaining the domain list, which requires someone to actually track the AI tool landscape on an ongoing basis.

A few things to watch for specifically:

• API traffic: Some employees, particularly in engineering and data science, will use AI services via API rather than browser. Look for outbound HTTPS connections to api.openai.com, api.anthropic.com, and similar endpoints. These often bypass browser-level controls entirely.
• Encrypted tunnel services: If someone is routing traffic through a personal VPN or SSH tunnel to avoid corporate network monitoring, that is a separate policy issue, but it is worth knowing whether your DLP tools can detect it.
• Volume anomalies: Large outbound data transfers to unfamiliar SaaS endpoints can indicate bulk document processing through an external AI tool.

One caveat: network telemetry tells you that a connection happened. It does not tell you what data was sent. For that, you need endpoint-level visibility or, more practically, the other methods below combined with policy enforcement.

Browser Extension Audit

This one is consistently underestimated. Browser extensions are one of the primary vectors for unauthorized AI tool usage, and they operate with permissions that most users never bother to read.

A 2023 study by Spin.AI found that over 50% of Chrome extensions had high or critical risk permissions, including the ability to read all data on all websites visited. AI-powered browser extensions are particularly concerning because their entire value proposition involves processing the content you are looking at. Grammar tools, writing assistants, meeting summarizers, email drafters: they all need access to the text on your screen to function.

If you are managing endpoints through Microsoft Intune, Google Workspace admin, or Jamf, you can pull a centralized inventory of installed browser extensions. Chrome Enterprise and Edge for Business both support extension allowlists and blocklists via group policy. If you have not configured these, now is the time.

When you pull the inventory, look for:

• AI writing assistants: Grammarly, Wordtune, QuillBot, Compose AI, Writesonic
• AI summarizers and readers: Smmry, TLDR This, Glasp, ChatGPT browser extensions (official and unofficial)
• Meeting AI tools: Otter.ai, Fireflies.ai, Fathom, tl;dv, Krisp
• Code assistants: GitHub Copilot (if not enterprise-licensed), Tabnine, Codeium, Amazon CodeWhisperer extensions

Pay special attention to extensions that are not from well-known vendors. The Chrome Web Store has seen a proliferation of thin wrapper extensions that route queries to OpenAI's API through unknown third-party servers. In June 2023, researchers at Guardio Labs identified several malicious ChatGPT-branded extensions that were harvesting Facebook session cookies. The attack surface here is real.

Expense Report and Procurement Review

People pay for the tools they find valuable. That makes your expense reports and corporate credit card statements a surprisingly effective discovery mechanism.

Run a keyword search across expense submissions and P-card transactions for the past 12 months. Search for "OpenAI," "ChatGPT," "Anthropic," "Claude," "Midjourney," "Jasper," "Copilot," "AI," and the names of any tools you have already identified through network or extension audits. Also search for common pricing tiers: $20/month is the ChatGPT Plus price point, $25/month is Midjourney's standard plan, and $10/month flags several writing tools.

This approach catches something network telemetry misses: tools used on personal devices. If an employee subscribes to an AI service and expenses it, they may be using it on a personal laptop or phone, outside your network entirely. That is a significant information governance gap, especially if they are processing work documents through it.

Procurement and accounts payable should also review any new SaaS contracts or trial agreements signed in the past year. Departments sometimes sign up for AI tools through free trials that auto-convert to paid subscriptions. The 2024 Zylo SaaS Management Index found that the average organization has 40% more SaaS applications than IT is aware of, and AI tools are a rapidly growing share of that gap.

Putting It Together: The Inventory and What Comes Next

Once you have data from all three sources, consolidate it into a single inventory. For each tool, document:

• The tool name and vendor
• Which employees or departments are using it
• What data types are likely being processed (client data, financial records, PHI, source code, legal documents)
• The vendor's data retention and training policies
• Whether the tool has a business or enterprise tier with contractual data protections
• Whether a BAA, DPA, or other required agreement is in place

This inventory becomes the foundation for a risk assessment. Some tools will be fine to continue using with proper agreements in place. Others will need to be blocked immediately. The important thing is moving from a state of not knowing to a state of informed decision-making.

For organizations subject to the FTC's recent enforcement actions around AI and data privacy (see the Rite Aid order from December 2023, which included a five-year ban on using AI-based surveillance systems), or the SEC's evolving expectations around AI risk disclosure, this kind of audit is not just good hygiene. It is increasingly a regulatory expectation.

How FirmAdapt Addresses This

FirmAdapt is designed to reduce the shadow AI problem by giving employees a compliant AI platform that actually works for their use cases. When people have access to capable, approved AI tools with proper data governance controls built in, the incentive to go find unauthorized alternatives drops significantly. FirmAdapt's architecture keeps data within defined boundaries, enforces retention policies, and maintains audit logs that satisfy regulatory requirements across healthcare, financial services, defense, and legal contexts.

Running a discovery audit will likely reveal that much of the unauthorized AI usage in your organization stems from legitimate productivity needs that your current toolset is not meeting. FirmAdapt gives you a way to meet those needs without the information governance gaps. It does not eliminate the need for ongoing monitoring, but it does change the conversation from "stop using AI" to "use this instead."