The Missing Layer
Early on, when I was testing whether AI could handle compliance work, I fed it a Business Associate Agreement and asked it to review. It came back with observations about formatting consistency, a note about a missing signature block, and a suggestion to “ensure compliance with applicable regulations.” Technically correct. Professionally useless. It read like a first-year associate who’d memorized the textbook but had never seen what a bad BAA actually costs someone.
Every practitioner I talk to has their own version of this story. Lawyers who asked AI to review a contract and got back a summary of what the contract said. Technologists who built retrieval systems that could find the right regulation but couldn’t tell you what it meant operationally. Business people who were promised AI would cut their compliance costs in half, then watched it produce work that still needed to be redone by a human.
The gap isn’t capability. The models are genuinely good at reading, reasoning, and following instructions. The gap is that nobody gave them the right instructions. Not a one-off prompt. Not a clever trick you type into a chat window. The accumulated judgment that separates a practitioner with ten years of experience from someone who just passed the bar.
That judgment is the missing layer.
What a Skill Actually Is
The AI ecosystem has developed a specific mechanism for encoding domain expertise: skills. A skill is a set of reusable instructions that teaches an AI system how a domain expert thinks about a specific task. Not fine-tuning the model. Not retraining it on your data. Writing down, in structured detail, what you check, in what order, and what “good” looks like at each step.
Think of it as the difference between handing someone a regulation and handing them a regulation plus ten years of annotations, failure modes, and “here’s what actually matters when you see this in the wild.”
Skills sit inside a broader extensibility stack. MCP servers give the AI access to external systems: your databases, your document repositories, your APIs. Plugins bundle skills and connections into packages that can be shared and installed. But skills are the layer where the expertise lives. The connections and infrastructure matter, but they’re plumbing. Skills are the judgment.
When I built Rote’s BAA Review skill, I wasn’t teaching the AI what 45 CFR 164.504(e)(2) says. It already knows that. I was teaching it what a compliance officer with a decade of BAA reviews checks first when one lands on their desk. Which provisions are the ones that actually kill deals. Which gaps show up in startup BAAs so often that I’ve stopped being surprised by them. That’s not information the model lacks. It’s prioritization the model hasn’t been given.
Building One
There’s no single right way to build a skill. Anthropic published a comprehensive guide to skill-building that covers the fundamentals: structure, frontmatter configuration, supporting files, testing, distribution. If you’re starting from zero, that guide is the place to begin. It’ll get you from “I’ve never built a skill” to “I have a working SKILL.md” in an afternoon.
What the guide gives you is the architecture. What it can’t give you is the domain expertise that makes a skill worth running. That part comes from you, and the path you take to get there depends on what you know and how you know it.
Here’s the path I took building Rote’s compliance skills, and why it worked for this domain.
Some of the skills I liked building the most started with the regulation itself. When you’ve done a task hundreds of times, you internalize the regulatory basis so completely that you forget which specific provision drives the instinct. The AI turned out to be genuinely useful for tracing those instincts back to their source, mapping my “check this first” reflexes to the actual CFR provisions that made them right. Starting from the regulation gave me a clean foundation. Then I could layer on what the regulation doesn’t tell you.
That second layer is where the real work of skill-building lives, regardless of which path you take. You write the instructions as if you were onboarding a sharp but inexperienced analyst. What does the textbook leave out? Where do the common mistakes cluster? What does the output need to look like for the person downstream who has to act on it? For BAA review, that meant encoding which provisions are the ones that actually kill deals, which gaps show up in startup BAAs so frequently that I’ve stopped being surprised by them, and what a finding needs to say so the client can act on it without a follow-up call.
A technologist building a code review skill would start from a completely different place. Maybe from the pull request template their team already uses, or from the checklist they run mentally every time they review someone’s code. A lawyer building an NDA triage skill might start from their firm’s classification criteria. The entry point changes. The principle doesn’t: you’re encoding what you know that the textbook doesn’t cover.
Structure matters more than most people expect. The order in which the skill checks things affects what it prioritizes. A skill that starts with administrative safeguards and ends with technical safeguards will weight its analysis differently than one that starts with technical safeguards. Neither is wrong in the abstract, but a skill can be fine tuned for your specific use case.
And the first version is never right. The gap between what you know and what you can articulate clearly enough for a machine to execute on is real and humbling. I went through more iterations on the BAA Review skill than I’d like to admit. The AI helped me see my own blind spots by forcing me to be explicit about things I’d been doing on autopilot.
Evaluating: Beyond Vibes
Most people evaluate AI output by reading it and deciding whether it “feels right.” That’s vibes. Vibes don’t scale and they don’t catch the subtle failures.
I evaluate a skill the same way I’d evaluate a junior associate’s work product. Not “does this sound smart” but “does this catch the thing an experienced practitioner would catch, and does it miss the things a textbook-trained person would miss?”
For the BAA Review skill, that means running it against BAAs with known issues. If the skill misses the subcontractor provision gap, the skill needs work. If it flags formatting issues but doesn’t catch a missing breach notification timeline, it’s doing the wrong job. The evaluation criteria aren’t abstract. They’re: would I send this to a client? If I wouldn’t, what’s missing?
This is where practitioners have an advantage that’s hard to replicate. You know what wrong looks like because you’ve lived with the consequences of wrong. An engineer can build the evaluation harness. They can’t define what the evaluation criteria should be, because that requires having been the person responsible when a bad BAA created a real problem.
Customizing Skills: A Critical Step You Need to Take
This is the section many people skip, and it’s the one that matters most.
I watch how people use skills in the wild. Most of them pull a skill from a marketplace or a repository, run it, and accept the output. This is the equivalent of downloading a contract template and sending it unsigned and unmodified. It works, technically. It’s not customized to your situation.
A lawyer knows you never send a template brief without adapting it to your jurisdiction, your judge’s known preferences, your client’s specific facts. A technologist knows you don’t deploy a library with its default configuration into production. A business operator knows that a playbook tuned for enterprise sales doesn’t work for SMB without significant rework.
Skills are the same. The defaults are someone else’s judgment calls about what matters most, what order to check things in, what “good” looks like. Those judgment calls may be excellent. They’re still not yours.
Here’s what customization actually looks like in practice.
Reweighting priorities. A compliance skill built for health systems might check administrative safeguards first. If you work with health tech startups, technical safeguards are where the gaps actually live. Same skill, different emphasis, meaningfully different output. The base skill doesn’t know your client mix. You do.
Adding institutional knowledge. The skill knows the regulation. It doesn’t know that a gap you find more often than you should in post-merger organizations is an outdated Notice of Privacy Practices. That’s your pattern recognition from years of doing this work. Adding it takes one line in the skill. The value it creates is disproportionate to the effort.
Adjusting output format. A skill that produces a fifteen-page report is perfect for a consulting engagement. For an internal audit committee that meets for ninety minutes and covers six agenda items, you need the executive summary and the findings table. Customization isn’t always about what the skill analyzes. Sometimes it’s about how the output lands with the person who has to act on it.
Removing what doesn’t apply. A BAA review skill built for the general case checks provisions that aren’t relevant to every engagement type. Trimming irrelevant checks isn’t laziness. It’s signal-to-noise discipline. A shorter, more focused output that addresses the actual risk profile is more useful than a comprehensive one that buries the findings that matter.
The meta-point is this: customization is where your specific experience gets encoded. The base skill is someone’s generalized expertise. Your customization layer is your particular expertise, the patterns you’ve seen, the failures you’ve lived through, the shortcuts you’ve earned. Two practitioners customizing the same underlying skill will produce meaningfully different results. Neither is wrong. Both reflect real, accumulated judgment.
If you’re using a skill without customizing it, you’re borrowing someone else’s judgment instead of encoding your own. For a first pass, that’s fine. As a permanent workflow, you’re leaving your own expertise on the table.
Who Captures the Value
The practitioners who encode what they know are the ones who multiply their capacity. Not the AI companies. Not the platform builders. The people who have done the work, know the failure modes, and can articulate what “good” looks like in a specific domain.
The infrastructure exists now: skills for encoding expertise, MCP servers for connecting systems, plugins for bundling and sharing it all. The hard part was never the infrastructure. The hard part is the expertise itself. And that’s yours.
A BAA review that took me forty-five minutes by hand takes minutes to review and confirm with the skill I built running in Claude. I didn’t get faster at reading. I got better at building skills that provide me the information required to confirm the analysis and conclusions of the LLM. The reading ability was always there in the model. What wasn’t there was my judgment about what to prioritize, what to flag, and what to ignore.
Make Them Yours
Rote’s compliance skills are open source on GitHub: BAA Review, gap analysis, framework mapping, risk assessment. All of them.
But if you read this piece and the section that stuck was the one about customization, here’s the invitation: don’t just run them. Take the BAA Review skill and reweight it for your client mix. Add the failure modes you’ve seen that I haven’t. Adjust the output for the stakeholder who’s going to read it. Strip out the checks that don’t apply to how you practice.
The base skills represent my judgment from my experience. Your customization layer is where your judgment starts compounding. The value isn’t in the skill I wrote. It’s in the version of it that reflects what you know.
If you’re a practitioner in any domain, the methodology transfers. You’ve done the same task hundreds of times. You know where the textbook stops and the real work starts. Encode that. Start with someone else’s skill if you want. Then make it yours.
Claude was my editor on this piece.