Your Agent Security Score is a Number, Not a Feeling

"Are we secure?"

If the answer starts with "I think so" or "we've done a lot of work on it," you don't actually know. You have a feeling. Feelings don't survive incident reviews.

Navil gives you a number. Run navil test, get a score out of 100, see exactly which attack vectors you're covered against and which ones you're not. No ambiguity. No gut checks. A number you can put in a dashboard, track over time, and fail a build on.

The Problem: Security Without Measurement

Most agent security today looks like this: you write some policies, block some obvious things, hope for the best. When someone asks "how secure are our agents?" you list what you've done, not what you've measured.

This is how application security worked before SAST and DAST tools made vulnerability scanning standard. Before those tools, security was a checklist. After them, security was a metric. The shift from checklist to metric changed everything — suddenly you could track progress, set thresholds, and hold teams accountable.

Agent security needs the same shift. Navil's coverage scoring is that shift.

The Navil-200: A Standardized Attack Benchmark

Navil ships with 200 attack simulations across the threat categories defined in our open threat taxonomy. We call it the Navil-200.

Each simulation reproduces a real attack pattern — prompt injection, tool poisoning, credential exfiltration, data leakage, privilege escalation, lateral movement — without generating real network traffic or touching real data. They're safe to run in production.

The 200 vectors break down across 11 SAFE-MCP attack categories:

• Prompt injection variants — 34 vectors covering direct injection, indirect injection via tool responses, multi-turn escalation, and encoding-based evasion
• Tool poisoning — 28 vectors simulating manipulated tool descriptions, hidden instructions in schemas, and tool substitution attacks
• Credential exfiltration — 24 vectors testing credential access patterns, exfil via tool responses, and token leakage
• Data exfiltration — 22 vectors covering staged data gathering, combination attacks, and covert channel construction
• Privilege escalation — 20 vectors testing permission boundary violations, role confusion, and capability expansion
• Lateral movement — 18 vectors simulating cross-server pivoting, cross-agent exploitation, and trust boundary violations
• Session manipulation — 16 vectors testing context poisoning, session hijacking, and state confusion
• Supply chain attacks — 14 vectors covering malicious dependencies, registry poisoning, and update hijacking
• Denial of service — 12 vectors testing resource exhaustion, infinite loops, and rate limit bypass
• Rug pull attacks — 8 vectors simulating post-install behavior changes and delayed payload activation
• Shadowing attacks — 4 vectors testing tool description manipulation to redirect agent behavior

Running the Test

One command:

navil test --pool default

That runs all 200 vectors against your current setup — your policies, your anomaly detection config, your threat blocklist. Takes about 60 seconds.

The output:

Navil Security Coverage Report
═══════════════════════════════════════════════════
Overall Score: 84.7 / 100

Category Breakdown:
  Prompt Injection      ████████████████████░  29/34  (85.3%)
  Tool Poisoning        █████████████████████░ 25/28  (89.3%)
  Credential Exfil      ████████████████░░░░░  18/24  (75.0%)
  Data Exfiltration     ███████████████████░░  19/22  (86.4%)
  Privilege Escalation  ████████████████░░░░░  15/20  (75.0%)
  Lateral Movement      █████████████████░░░░  15/18  (83.3%)
  Session Manipulation  ████████████████████░  14/16  (87.5%)
  Supply Chain          █████████████████████░ 13/14  (92.9%)
  Denial of Service     █████████████████████░ 11/12  (91.7%)
  Rug Pull              █████████████████████░ 7/8    (87.5%)
  Shadowing             █████████████████████░ 4/4    (100.0%)

Gaps Found: 31 vectors unblocked
  → Run 'navil test --show-gaps' for details
  → Run 'navil test --fix-suggestions' for remediation steps

Time: 58.3s | Vectors: 200 | Blocked: 170 | Unblocked: 30

You know exactly where you stand. 84.7% isn't a feeling — it's a measurement. And the 15.3% gap isn't a mystery — it's a list of specific attack vectors with specific fixes.

Understanding the Gaps

The gaps are the valuable part. Run:

navil test --show-gaps

Output:

Unblocked Vectors (31):
───────────────────────

CRED-EXF-07: Credential access via delegated sub-agent
  Category: Credential Exfiltration
  Severity: Critical
  Attack: Sub-agent requests parent credentials through delegation chain
  Fix: Add delegation depth limit to policy.yaml
       delegation:
         max_depth: 2
         inherit_permissions: false

PRIV-ESC-14: Permission expansion via tool chaining
  Category: Privilege Escalation
  Severity: High
  Attack: Agent chains read-only tool with write tool to bypass restrictions
  Fix: Add tool-chain policy rule
       rules:
         - deny:
             sequence: ["fs_read", "http_post"]
             when: data_sensitivity >= "medium"

... (29 more)

Each gap tells you: what the attack is, how severe it is, and exactly what policy rule to add to block it. You don't need to be a security expert. You need to copy YAML.

Fix Suggestions: From Gap to Policy

Want Navil to write the fixes for you?

navil test --fix-suggestions > fixes.yaml

This generates a YAML file with every policy rule needed to close every gap. Review it, merge it into your policy.yaml, run navil test again. Your score goes up.

The workflow:

1. navil test → see your score
2. navil test --show-gaps → see what's missing
3. navil test --fix-suggestions > fixes.yaml → get the fixes
4. Review and merge into policy.yaml
5. navil test → confirm improvement

Repeat until you're at the coverage level you need.

CI/CD Integration: Fail the Build

The real power of a numeric score is automation. Add this to your CI pipeline:

navil test --threshold 90

If coverage drops below 90%, the command exits with code 1. Your build fails. No human needs to notice that someone removed a policy rule or added an insecure MCP server config — the pipeline catches it.

For GitHub Actions:

- name: Navil Security Gate
  run: |
    pip install navil
    navil test --threshold 90 --output sarif > navil-results.sarif
 
- name: Upload SARIF
  uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: navil-results.sarif

The SARIF output integrates with GitHub's Security tab. Every unblocked vector shows up as a security finding — with severity, description, and remediation steps. Your security team sees agent security gaps alongside traditional code vulnerabilities.

For GitLab CI:

navil-security:
  stage: test
  script:
    - pip install navil
    - navil test --threshold 90 --output json > navil-results.json
  artifacts:
    paths:
      - navil-results.json
    when: always

Tracking Over Time

Your security score should go up over time, not down. Navil's dashboard tracks your score history so you can see trends:

• Score after initial setup
• Score after adding policies
• Score after enabling anomaly detection
• Score after joining the community threat network

Each layer of Navil's security stack adds coverage. Token scoping reduces your attack surface. Policies block known-bad patterns. Anomaly detection catches behavioral threats. The threat network provides community intelligence. Each one pushes your score higher.

If your score drops, something changed. A policy was removed. A new MCP server was added without governance. A config was modified. The score delta tells you something regressed — and --show-gaps tells you exactly what.

Custom Attack Vectors

The Navil-200 covers the standard threat landscape, but your setup might have unique risks. Add custom vectors:

# custom-vectors.yaml
vectors:
  - name: "CUSTOM-01: Internal API exfiltration"
    category: data_exfiltration
    severity: critical
    simulate:
      tool: http_client
      target: "internal-api.company.com/secrets"
      expected: blocked
    remediation: "Add domain deny rule for internal-api.company.com"

Run with custom vectors included:

navil test --pool default --include custom-vectors.yaml

Your score now reflects both standard and organization-specific coverage.

The Score is the Starting Point

An 84.7% score means you're blocking 170 out of 200 known attack patterns. That's good. But security isn't a destination — it's a practice.

The Navil-200 grows as new attack patterns are discovered. The community threat network feeds new vectors back into the benchmark. Your score might drop when new vectors are added — that's a feature, not a bug. It means a new threat was discovered and you now know whether you're protected against it.

The workflow is simple: measure, identify gaps, close gaps, measure again. No feelings. No guesswork. Just a number that tells you where you stand.

Getting Started

Install Navil and find out your score:

pip install navil

Run your first test:

navil init
navil test --pool default

Set your threshold and add it to CI:

navil test --threshold 85

Check your gaps:

navil test --show-gaps --fix-suggestions

What's Next

You've got a score. You've got policies. You've got anomaly detection and community threat intelligence. But what about the credentials your agents use? In the next post, we'll cover Navil's credential governance — how to stop API keys from living in plaintext MCP config files and start managing them like actual secrets.

200 attack simulations. 11 categories. One number. Stop guessing about your agent security. Start measuring it.

Install Navil

Measure your agent security coverage with standardized attack simulations:

pip install navil

Visit navil.ai to explore the dashboard, track your score over time, or join the community. Open source. Agent-native. Security you can quantify.