Autotuned Guardrails

Generate a recommended Dome configuration directly from a Vijil evaluation.

TL;DR: Evaluate your Agent with Vijil, then call Dome.create_from_vijil_evaluation(evaluation_id, api_key) to build a Guardrail configuration tuned to the vulnerabilities that evaluation found. Optionally pass a latency_threshold to trade coverage for speed.

Rather than hand-picking Guards, you can let a Vijil evaluation drive your Dome configuration. This tutorial evaluates an unprotected Agent, generates a recommended configuration from the results, and re-evaluates the protected Agent to confirm the improvement.

The full runnable code is in the AutotunedGuardrails tutorial folder.

Prerequisites

Install the dependencies and set your keys:

pip install vijil-dome vijil

OPENAI_API_KEY=<your OpenAI key>
VIJIL_API_KEY=<your Vijil API key>

Obtain a Vijil API key from evaluate.vijil.ai. If you are on the free plan, you also need an Ngrok authtoken (NGROK_AUTHTOKEN), which Vijil uses to create a private HTTPS endpoint to your Agent. Premium users can skip this.

Walkthrough

1. Evaluate the Baseline Agent

   Run safety and security harnesses against your unprotected Agent. When the evaluation completes, note its evaluation ID from the logs or from the Vijil Console.

   python -m evaluate_baseline_agent
   
2. Build Guardrails From the Evaluation

   Dome.create_from_vijil_evaluation fetches the recommended configuration for that evaluation and returns a ready-to-use Dome:

   from vijil_dome import Dome
   
   dome = Dome.create_from_vijil_evaluation(
       evaluation_id=YOUR_EVALUATION_ID,
       api_key=VIJIL_API_KEY,
   )
   

   Wrap your Agent so every request passes through the tuned Guardrails:

   class ProtectedAgent:
       def __init__(self, agent, dome: Dome):
           self.agent = agent
           self.dome = dome
   
       async def guardrailed_invoke(self, prompt: str) -> str:
           input_guard_result = await self.dome.async_guard_input(prompt)
           if input_guard_result.flagged:
               return "Input was flagged by Vijil Dome. This request violates the agent's usage policy."
   
           agent_response = await self.agent.invoke(prompt)
   
           output_guard_result = await self.dome.async_guard_output(agent_response)
           if output_guard_result.flagged:
               return "This output was blocked by Vijil Dome."
   
           return agent_response
   

   Re-run the evaluation against the protected Agent:

   python -m evaluate_protected_agent --evaluation-id=YOUR_EVALUATION_ID
   
3. Optimize for Latency

   Pass a latency_threshold (in seconds) to bias the recommendation toward faster Guards:

   python -m evaluate_protected_agent --evaluation-id=YOUR_EVALUATION_ID --latency-threshold=1.0
   

   In code, the same argument is available on the factory:

   dome = Dome.create_from_vijil_evaluation(
       evaluation_id=YOUR_EVALUATION_ID,
       api_key=VIJIL_API_KEY,
       latency_threshold=1.0,
   )
   

Troubleshooting

Some Windows antivirus tools flag Ngrok. Ngrok is safe, and Vijil guarantees that traffic reaching your Agent through Ngrok originates only from Vijil services, so your Agent is never publicly exposed.

Next Steps

Configure Guardrails

Fine-tune the recommended configuration

Understand Results

Read the evaluation that drives the configuration