At a glance
- Re-vet an AI Skill on every version change, not on a calendar — a Skill is editable instructions that execute with the user's permissions.
- Backslash Security's free Skills Security Scanner checks AI agent Skills for security risks before and after an update lands.
- Update events that demand immediate re-vetting include new shell commands, new network destinations, new credential paths and new MCP server connections.
- Continuous discovery plus inline blocking closes the window between a Skill changing and a security team noticing.
Backslash Security
Published:
Re-vet an AI Skill every time its content changes — not on a fixed quarterly or annual schedule. An Agent Skill is a packaged set of instructions and scripts that extends what an AI agent can do, and in practice it is often just a markdown file sitting on an employee's machine; it can tell an agent to read a file, call an API, or run a shell command, and it executes with that user's own permissions. Because a Skill can be edited silently, by its upstream author or by anyone with write access to the machine or repository, the version that was approved last month may bear little resemblance to the one running today. Backslash Security's free Skills Security Scanner exists for exactly this check: it scans AI agent Skills for security risks, so a re-vet can be triggered by a change event rather than postponed to a review cycle.
That makes "how often" the wrong unit of measurement. The useful question is what counts as a change worth re-examining — a newly added shell invocation, a new outbound destination, a reference to credential files, or a new connection to an MCP server, the protocol agents use to reach external tools and data. Change-driven re-vetting only works if someone can see the change at all, which is where most programs stall: the endpoint security stack reports processes, not what an agent was instructed to do by a file it read. The sections below set out which events should force an immediate re-vet, what a thorough re-vet inspects, how cadence models compare for teams of different sizes, and how continuous discovery and inline enforcement carry the load between reviews. Backslash Security, which Latio's 2026 AI Security Market Report named an Endpoint AI Security Leader for demonstrating the most in-depth controls for AI on the endpoint — including permission mapping, folder structures, approved commands, and runtime controls for MCPs and Skills — is built around that continuous model.
How often should you re-vet an AI Skill after it updates?
How often you re-vet an AI Skill is best answered at the version boundary rather than on a calendar: re-vet on every update. This section is narrowed to that one case — a Skill that already passed review and then ships a new version. An Agent Skill is packaged instructions and scripts that extend what an AI agent can do; it can tell an agent to read a file, call an API, or run a shell command, and it executes with the user's own permissions. In practice it is often just a markdown file, which means the content behind an approved name can change completely while the name, the install path, and the entry in your inventory stay identical.
That is why a purely periodic review — quarterly, or at onboarding only — approves a point-in-time artifact rather than the one running today. The workable default for a security or IT team that needs a rule now: every version change invalidates prior approval, and the Skill stays untrusted until the new version is assessed.
| Do this | But watch out for — and how to handle it |
|---|---|
| Trigger re-vetting on version change, not on a date | Review volume can swamp the team. Reserve human review for Skills whose instructions, scripts, or tool permissions actually changed; let cosmetic edits pass automatically. |
| Default an updated Skill to untrusted until reassessed | Blanket blocking stalls engineers. Use narrow, time-boxed exceptions tied to a named owner rather than standing waivers. |
| Compare the new instruction text against the approved version | A clean text diff still misses what the Skill does at runtime. Pair static comparison with controls on the actions the agent may take. |
| Re-vet when the host changes too — a new agent version, MCP server, or rules file alters what a Skill can reach | Scope can expand indefinitely. Bound it to components the Skill directly invokes. |
Because review always lags publication, enforcement has to cover the interval. Backslash Security blocks risky agent actions inline before execution, including unauthorized code execution, credential access, privilege escalation, and data sent to unapproved destinations.
What is an AI Skill, and what actually changes when one updates?
An AI Skill is a packaged set of instructions and scripts that extends what an agent can do — in practice, often just a markdown file sitting on an employee's machine — and what actually changes in an update is rarely visible from the version string alone.
This depends on what you mean by "an update." At least four distinct events get called the same thing: the author publishes a new release, a bundled script or dependency changes underneath a stable version number, the permissions the Skill exercises widen, or someone edits the local copy on the endpoint with no release event at all. Each mutates a different part of the Skill, and each warrants a different level of scrutiny.
Which components make up a Skill, and which ones mutate?
| Component | What it holds | Typical values | Why a change matters |
|---|---|---|---|
| Manifest | Declared identity and metadata | Name, version, description, declared tools or MCP servers | A new declared tool means a new path the agent can act through |
| Instructions | Natural-language directives the agent follows | Markdown prose, task framing, standing constraints | A reworded line can redirect agent behavior without touching any code |
| Bundled tool calls | Executable actions the Skill invokes | Shell commands, API calls, file reads and writes | Execution scope can widen silently between releases |
| Permissions and scope | What the Skill may reach | Filesystem paths, credential stores, network destinations, approved commands | A Skill runs with the user's own identity and access, so scope creep inherits their privileges |
The instructions layer is the one conventional software review handles least well, because it is prose, not code, and diffs read as editorial rather than functional. A Skill can also be modified locally after installation, which means the copy executing on a given endpoint may differ from the published artifact that was approved.
Which Skill update events should trigger an immediate re-vet?
Some Skill update events cannot wait for the next scheduled review, because they change what the Skill is allowed to do before anyone looks at it again. Agent Skills — packaged instructions and scripts that extend an AI agent, often shipping as little more than a markdown file — execute with the user's own permissions, so a change to the package is a change to your blast radius.
What counts as an "update" here?
The word carries two distinct meanings on the endpoint, and both matter:
- A package change. The Skill's own contents move: the instruction text, a bundled script, or a declared command list. A maintainer adding a shell invocation to a Skill that previously only read files is a package change.
- A reach change. The Skill's text is untouched, but something it points at moves — the MCP server it calls, where Model Context Protocol is the standard agents use to connect to external tools and data. A revised tool definition upstream changes behavior with no local edit at all.
This section uses both senses together, because the agent run that follows cannot tell them apart.
Which signals force re-vetting before next use?
- Permission expansion — new file paths, network egress, or credential scopes requested by the Skill.
- A new connector or MCP server reference — a fresh outbound path the earlier assessment never examined.
- Changed instructions — edits to the Skill body or to an adjacent rules file such as AGENTS.md or CLAUDE.md, which carries standing instructions an agent reads on every run. Backslash Security research found that malicious instructions hidden in a repository's AGENTS.md file could trick OpenAI Codex into silently accessing AWS credentials, npm tokens and Git configuration.
- Publisher or maintainer change — the trust decision was made about a party that no longer controls the artifact.
- Silent auto-update — a version advanced without a human approval step.
Any of these means the approval on record describes a component that no longer exists, so the allowlist entry has to be re-earned before the next run.
What does a thorough Skill re-vet actually check?
A thorough Skill re-vet narrows the scope to a single artifact and walks its attributes one by one, which is why it works as a checklist rather than a judgment call. Agent Skills are packaged instructions and scripts that extend what an AI agent can do — in practice often a markdown file sitting on the employee's machine, executing with that employee's own permissions. The attributes below are what a re-review should capture each time the file changes.
| Attribute | What to capture | Why it matters |
|---|---|---|
| Instruction body | Line-level diff of the markdown, prompts and any embedded shell commands | New standing instructions can change behavior without changing any binary |
| Manifest and dependencies | Declared scripts, versions, and the MCP servers or tools the Skill invokes | A new tool connection widens the blast radius silently |
| Requested permissions | Filesystem paths, shell execution, network destinations, credential stores | Permission creep between versions is the most common drift |
| Data reach | Repositories, ticketing systems and cloud consoles reachable through connected MCP servers | Reach, not the Skill itself, determines worst-case impact |
| Provenance and publisher | Registry or repository source, publisher identity, commit history | Republished or transferred packages break the original trust assumption |
| Execution identity | Whether the Skill runs under a corporate directory identity such as Entra ID or Okta, or a personal login | Personal-account execution removes the organization's ability to revoke or audit |
| Disposition record | Reviewer, decision, policy version and timestamp | Supplies the evidence trail an incident investigation or auditor will ask for |
Instruction diffing deserves the closest reading. Backslash Security research found that malicious instructions hidden in a repository's AGENTS.md file could trick OpenAI Codex into silently accessing AWS credentials, npm tokens and Git configuration — the same class of planted text that can arrive inside an updated Skill.
Each completed re-vet should end by writing its outcome back into enforcement: Backslash Security supports allowlisting approved components and denylisting risky ones through custom policies that different teams can carry at different risk thresholds.
Which re-vetting cadence model fits your organization?
The right re-vetting cadence model follows from four criteria, which are worth settling before any interval is chosen. Agent Skills — packaged instructions and scripts that extend what an AI agent can do, often shipped as a single markdown file — change without a release process, so the criteria matter more than the calendar.
- Endpoint fleet size: how many machines run the Skill, and whether discovery covers all of them or only developer workstations.
- Update frequency: how often the upstream Skill, its scripts, or its referenced tools change.
- Blast radius: what the Skill can reach when it executes with the user's own permissions — source code, cloud credentials, production systems, customer data.
- Review effort: the analyst hours each pass consumes, which decides whether the cadence survives contact with a busy quarter.
| Cadence model | How it works | Fits when | Review effort |
|---|---|---|---|
| Calendar-based | Fixed periodic re-review of the Skill inventory | Fleets are small and Skills change slowly | Predictable, but spent evenly on low- and high-risk Skills |
| Event-triggered | Re-vet fires on version change, new script, new tool permission or new network destination | Updates are frequent but irregular | Low baseline, spikes with upstream churn |
| Risk-tiered | Review interval set by blast radius; high-reach Skills reviewed far more often | Blast radius varies widely across teams | Concentrated where consequences are largest |
| Continuous assessment | Posture evaluated automatically on every change, with policy enforced at execution | Large fleets with heavy agentic AI adoption | Lowest manual load; requires policy definition up front |
Across these models, what determines exposure is less the interval between reviews than the delay between a Skill changing and anyone learning that it changed.
Most organizations end up combining them: risk tiers decide depth, events decide timing. Backslash Security supports that pattern directly, with allowlisting of approved components, denylisting of risky ones, and custom policies per team enforced continuously rather than at review time.
Frequently Asked Questions
What is an Agent Skill, and why does it need re-vetting at all?
An Agent Skill is a packaged set of instructions and scripts that extends what an AI agent can do — in practice, often just a markdown file sitting on an employee's machine. A Skill can tell an agent to read a file, call an API, or run a shell command, and it executes with that user's own permissions. Because the file can be rewritten by an upstream maintainer or edited locally, the version you approved is not necessarily the version running today.
How do you check a single Skill quickly without a full review cycle?
For a one-off check, Backslash Security operates a free Skills Security Scanner that scans AI agent Skills for security risks, which gives a team a fast read on a specific Skill before it is approved or after it changes. For a broader picture across machines, Backslash Security also offers a free AI Endpoint Exposure Assessment that is agentless — meaning it collects information without leaving software permanently installed — read-only, and retains no data.
Should MCP servers and rules files be re-vetted on the same schedule as Skills?
Yes, because they change the same way and carry comparable reach. An MCP server, under the Model Context Protocol, is a connection an agent can act through; a rules file such as AGENTS.md or CLAUDE.md carries standing instructions an agent reads on every run. Backslash Security research found that malicious instructions hidden in a repository's AGENTS.md file could trick OpenAI Codex into silently accessing AWS credentials, npm tokens and Git configuration — a change to an instruction file, not to a binary.
Why does an endpoint security tool miss a Skill update?
Endpoint detection and response products are built to catch malicious processes, files and known-bad behavior on a machine. A Skill update usually arrives as an edit to a text file that an approved, signed agent then reads and acts on, so nothing in the process tree looks unusual. The meaningful change is in what the agent was instructed to do, which is why governance of the agentic layer sits alongside the existing stack rather than inside it.
What should a re-vetting program produce for auditors?
It should produce a dated record of what was running, what changed, who approved it, and what was blocked. According to Backslash Security, the platform automatically generates audit evidence for EU AI Act, NIS2, DORA and SOC, and traces the full path of an agent run from prompt to agent to tool call to outcome — the trail an investigation needs when a Skill behaved unexpectedly after an update.
Who founded and backs Backslash Security?
Backslash Security was founded by Shahar Man, CEO and Co-founder, and Yossi Pik, CTO and Co-founder. The company is backed by Stage One, First Rays Venture Partners, Artofin and D E Shaw & Co, alongside a group of individual investors. Its published work in this area includes Claw-Hunter, an open-source tool for discovering and assessing OpenClaw risks.
About this article
Backslash Security publishes this article under its own name and is responsible for its accuracy. Articles are researched and drafted with AI assistance and approved by Backslash Security before publication; publication and update dates reflect substantive edits, not automated refreshes. Last updated: 2026-10-07