Building Guardrails
That Don't Kill
Latency

Performance + Safety in Production Agents
⚠️ "Our guardrails add 2.5 seconds to every request"

The Problem

With Sync Guardrails
2.5s
Per Request
With Async Guardrails
800ms
Per Request
Request
→
PII Check
(400ms)
→
Policy
(300ms)
→
Agent
(800ms)
→
Response
Total Latency: 2.5 seconds (Users expect <1 second)

Wrong vs Right Approach

❌ What Most Teams Do (Synchronous)
Request
→
All Guardrails
(700ms)
→
Agent
(800ms)
→
Response
Total: 1.5s + waiting time = Poor UX
✅ What You Should Do (Async + Streaming)
Request
→
Light Checks
(50ms)
→
Agent
(streaming)
→
Response
+ Async Checks
User sees response in 850ms. Guardrails run in parallel.

Where Guardrails Belong

Pre-Agent
(Synchronous)
  • Input validation
  • Rate limiting
  • Cost checks
  • Obvious violations
Post-Agent
(Asynchronous)
  • PII detection
  • Content filtering
  • Compliance checks
  • Logging & audit
Human-in-Loop
(Triggered)
  • High-risk actions
  • Edge cases
  • Escalations
  • Audit flags

PII Detection Strategy

Sync: Quick Pattern Matching (50ms)
What to Check
Obvious patterns (SSN, CC)
When to Block
High-confidence matches
Async: Deep Analysis (200ms)
What to Check
Names, emails, addresses
When to Intervene
Flag for review/redact
Streaming: Real-time Intervention
How it Works
Check tokens as they stream
Action
Stop stream if PII detected

Policy Enforcement Patterns

Pre-Flight Checks (Sync)
Cost/Rate Limits
Block before processing
Banned Content
Keyword blocklists
Output Analysis (Async)
Content Safety
Toxicity, bias detection
Compliance
Industry regulations
Circuit Breakers (Triggered)
Anomaly Detection
Unusual patterns
Auto-Escalate
Human review

The Results

Before (Sync)
Average Latency
2.5s
P95 Latency
4.2s
User Satisfaction
64%
After (Async)
Average Latency
850ms
P95 Latency
1.2s
User Satisfaction
91%
Separate what MUST be sync from what CAN be async.
⚡
Guardrails Patterns
community.nachiketh.in
🎓
Production Systems Bootcamp
bootcamp.nachiketh.in