Adopting AI tooling
without losing track of it
For organizations and individuals about to start using generative AI on regulated work. The goal is to arrive eighteen months from now with tools that were registered, owned, and verified from the first day, rather than reconstructed afterward.
Then your problem is different, and it is bigger than a form. AI capability enters five ways and only one of them is a purchase, so finding what is already running means probing the four paths that appear in no vendor list, no spend export, and no agreement register. That work is published as an open-source skill, built to complete a first pass on inputs you can obtain under your own authority, without an administrator export and without filing a ticket.
Use the AI Tooling Inventory skill →1. Decide whether you can specify a correct output
This comes before the tool selection, and in regulated work it decides everything downstream. In most businesses a wrong output is a bad experience. In this one it is a finding.
The concrete version: take ten to twenty documents from the workflow you want to automate that you have already reviewed by hand. Weight the set toward the ambiguous cases and the ones where two reviewers disagreed, because those are the cases that separate a tool that works from one that performs well on the easy half. That set is your graded set, and every tool you ever adopt for this workflow gets measured against it.
If you cannot assemble that set, you have found the question worth answering before you talk to a single vendor. A workflow whose correct output nobody can specify is not ready to be automated by anyone, at any price.
2. Choose the route, per workflow
There are three, and the choice is made per workflow rather than once for the organization. Most organizations end up using all three, and that is the healthy outcome rather than a mess.
- Buy
- Commercial software. You are purchasing a vendor's opinion about how your workflow should go, along with their roadmap, their model choices, and their subprocessor chain. Fastest to stand up, least control over what changes.
- Compose
- You assemble it yourself on top of a general model: prompts, a document store, a retrieval step, a review interface. Cheapest and most controllable. The cost moves out of construction and into maintenance and verification, which are the two line items organizations are least practiced at budgeting.
- Commission
- You pay a builder. You are purchasing build capacity and inheriting their verification discipline, or their lack of it. Worst of the three if you cannot grade the output, because you will get something that performs beautifully on the demo set with no way to know the month it stopped being right.
3. Watch the three ways AI arrives without a decision
Step 2 describes how you decide to add AI. Most of what an organization ends up running arrives without anyone deciding anything, along three routes that leave no transaction behind for procurement to catch. Starting from zero is the one moment when getting ahead of these is easy.
- A feature switched on in a product you already approved
- The EHR adds summarization. The transcription vendor adds a scribe. The document system starts suggesting classifications. Approval of the product gets read as approval of the feature, and it typically operates on the most sensitive data you hold, because it was added to the system that already held it.
- What to do now: ask each vendor, in writing, which AI features are enabled on your tenant, what data those features process, and which subprocessors are involved. Record each answer as its own row rather than as a footnote on the parent product.
- An integration attached to a product you already approved
- A plugin, a connector, a marketplace app. There is a real outside company here receiving real data, and often no agreement with them, because the gate was cleared by the platform rather than by what got connected to it. Approving a platform does not extend that platform's agreement to a different company connected to it.
- What to do now: set your identity provider to require administrative approval for third-party app consent before anyone starts, then review the app directory inside each platform. Doing this first is far cheaper than revoking grants later.
- A free third-party tool somebody signs up for
- A browser extension that summarizes what is on screen, a free web tier, a consumer app reached with a work email. A genuine external processor, no spend line to catch it, no questionnaire, no contract, and the data leaves the building the first time it is used.
- What to do now: decide the rule while nobody has broken it, say it plainly, and give people a sanctioned tool that does the thing they would otherwise reach for. Ask what people are using with no consequence attached. An organization that treats the answer as a disciplinary matter gets a clean inventory and a false one.
The obvious advice is to pull a tenant-wide report of every third-party app anyone has authorized. That is the right data, and it needs administrator access to your identity provider, which most people reading this do not have. So the advice becomes "file a ticket," and a ticket is not a delay. It is an abandonment: the work queues behind someone else's priorities and nobody finds out later why it never happened.
These three need nobody's permission and take an afternoon between them:
- Your own connected applications. Every major platform lets an individual see the third-party apps their own account authorized and the permissions each holds. In Microsoft that is the My Apps portal, under each application's permissions view. Google exposes the equivalent on the account's linked-apps page.
- Your own browser extensions.
chrome://extensionsoredge://extensions, filtered for anything that reads page content or the clipboard. - Your own inbox. Search for "verify your email", "welcome to", and "confirm your account". This catches what the connected-apps view structurally cannot: a tool signed up for with an email address and a password never created an authorization grant, so the signup confirmation is the only trace it left inside your walls.
You are looking at one account rather than the whole organization, so treat the result as a floor rather than a count. The tenant-wide export belongs in a second pass, once you have something concrete to justify the ask.
4. Three things on day one, for every tool
- Register it. One row, at the moment it starts being used rather than at the first audit. The register below is a place to start.
- Name one owner. An individual, not a department. The owner owes three things: the eval run, the change log, and the answer to whether the tool should still exist. That last one retires more tools than people expect.
- Grade it before you rely on it. Run the graded set from step one. Record the result. That number is what makes a later change visible as a change rather than as a feeling that something is off.
These are minutes of work at adoption and days of work in reconstruction. That difference is the entire argument for doing it now.
5. The register you start keeping
Runs entirely in your browser. Nothing is transmitted, stored on our servers, or seen by us. Put tool names and owner names in it, not organizational data.
| Tool | What it does | How it arrived | ePHI | BAA | Named owner | Last graded |
|---|
- How it arrived
- Licensed, built in-house, or commissioned per step 2, plus the three from step 3: feature switched on, integration attached, and free tool. The last three are the rows people forget, because nothing about them felt like adopting a tool. Two of them create an outside processing relationship on their own.
- ePHI
- Whether the tool creates, receives, maintains, or transmits electronic protected health information. Pasting a PHI-bearing document into a general assistant counts. 45 CFR 164.308(a)(1)(ii)(A) and OCR's risk analysis guidance attach to the data rather than to how the system was acquired.
- BAA
- Whether a Business Associate Agreement is in place with whoever actually operates the tool. For an integration that is the company that built the integration, not the platform it attaches to. For a build of your own it is the underlying model provider. "N/A" is a legitimate answer when no ePHI is involved. It is not a legitimate answer once the ePHI column says yes.
- Named owner
- One person, not a department. The owner owes the eval run, the change log, and the answer to whether the tool should still exist.
- Last graded
- The last time the tool was run against your graded set from step one. Same set, same scoring, every tool.
Four questions per row, once a quarter:
- Does it still pass the graded set? Same set, same scoring. A drop is a finding.
- Does it still have a named owner, and is that person still here?
- Is the compliance surface still current? The agreement, the subprocessor list, the model version, and for anything connected, the scopes it still holds. A vendor's underlying model can change between reviews with no contract amendment.
- Does it still need to exist?
This guide is a structure for your own adoption and record-keeping. It does not read anything, score anything against a regulatory standard, or produce a risk analysis. A HIPAA risk analysis under 45 CFR 164.308(a)(1)(ii)(A) requires considerably more than an asset list, and the asset list is its first step rather than its deliverable.
This guide is not:
- A risk analysis or a substitute for one
- A guarantee of HIPAA compliance
- Legal advice
For educational purposes only. Your entries stay in your browser. Nothing is sent to us.