Your First Policy

Overview

This guide walks you through creating a governance policy from scratch -- from deciding what to govern, to defining rules, binding the policy to an integration, and testing it with a live evaluation. By the end, you will have a working policy that detects PII in AI responses and blocks them from reaching users.

Step 1: Plan Your Policy

Before opening the dashboard, decide two things:

1. What are you governing? For this guide, we will detect personally identifiable information (PII) in AI-generated responses -- specifically Social Security Numbers and harmful content.

2. Which rule types do you need?

   • Deterministic rules use field-level conditions with operators (CONTAINS, MATCHES, EQUALS, etc.). They evaluate in milliseconds with no external calls. Use them for pattern matching, keyword detection, and structured checks.
   • Semantic rules use an LLM to evaluate content against a natural language prompt. They handle nuance -- tone, intent, context -- that field operators cannot express. Use them for subjective or context-dependent checks.

Most real-world policies combine both types. Deterministic rules catch the obvious cases fast; semantic rules catch the subtle ones.

Step 2: Create the Policy in the Dashboard

1. Navigate to Policies in the sidebar.
2. Click Create Policy.
3. Enter the policy details:
   • Name: PII Detection
   • Description: Detects and blocks PII such as Social Security Numbers in AI responses. Also flags harmful content where the AI oversteps its role.
4. Set the enforcement mode to MONITOR_ONLY for now (we will change this later).

Enforcement modes control what happens when a violation is detected:

• BLOCK -- Prevent the action from proceeding.
• APPROVAL_REQUIRED -- Hold the action for manual review before allowing it.
• WARN -- Allow the action but flag it for review.
• MONITOR_ONLY -- Log silently for audit purposes with no user-facing impact.

Step 3: Add a Deterministic Rule

Deterministic rules are the fastest layer of the evaluation engine. They check structured conditions against the input fields.

Add your first rule to catch Social Security Number patterns:

1. Click Add Rule inside your new policy.
2. Set the rule type to DETERMINISTIC.
3. Configure the rule:
   • Name: SSN Detection
   • Severity: CRITICAL
4. Add a condition:
   • Field: content
   • Operator: MATCHES
   • Value: \d{3}-\d{2}-\d{4}

This rule fires whenever the content field in the evaluation input matches the SSN pattern (three digits, dash, two digits, dash, four digits). Because it uses a regex operator, it catches any SSN-formatted string regardless of surrounding text.

You can add multiple conditions to a single rule. All conditions must match for the rule to fire. For example, you could add a second condition to only flag SSNs when the role field equals assistant, so user messages are not flagged.

Step 4: Add a Semantic Rule

Semantic rules bring LLM-powered evaluation for checks that cannot be expressed as field conditions.

Add a second rule to detect harmful AI responses:

1. Click Add Rule again.
2. Set the rule type to SEMANTIC.
3. Configure the rule:
   • Name: Harmful Content Check
   • Severity: HIGH
   • Prompt: Does this content contain medical advice, legal recommendations, or financial guarantees that the AI should not be making? Flag if the AI is overstepping its role by presenting opinions as facts, making promises about outcomes, or giving specific professional advice that requires a licensed practitioner.

The prompt is sent to the LLM along with the evaluation input. The LLM determines whether the content violates the prompt and returns an explanation. Unlike deterministic rules, semantic rules can understand context, detect hedging vs. assertion, and distinguish between informational content and prescriptive advice.

You can optionally attach a knowledge base to a semantic rule via the knowledgeBaseId field. This grounds the LLM evaluation in your organization's own documents using RAG -- useful for checking content against internal guidelines, product documentation, or regulatory text.

Step 5: Bind the Policy to an Integration

Policies are only evaluated when they are bound to an integration. Policy bindings control exactly which policies apply to which integration endpoints.

1. Navigate to Integrations in the sidebar.
2. Select the integration you want to govern (e.g., your REST API or Agent integration). If you have not created one yet, see the Quickstart guide.
3. Open the integration's Policy Bindings section.
4. Click Bind Policy and select your PII Detection policy.

When an evaluation request arrives via that integration's API key, all bound policies are automatically evaluated against the input. You can bind multiple policies to a single integration, and bind a single policy to multiple integrations.

Step 6: Test Your Policy

Use the evaluate endpoint to send test content and see your policy in action.

Test the deterministic rule

curl -X POST https://api.aguardic.com/v1/evaluate \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": {
      "role": "assistant",
      "content": "Sure, my SSN is 123-45-6789. Let me know if you need anything else."
    },
    "targetKey": "pii-test"
  }'

Expected response (with MONITOR_ONLY enforcement, the outcome is still ALLOW but violations are recorded):

{
  "success": true,
  "statusCode": 200,
  "data": {
    "outcome": "ALLOW",
    "enforcementAction": "ALLOW",
    "evaluationRunId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "sessionId": null,
    "reviewRequestId": null,
    "pollUrl": null,
    "violations": [
      {
        "id": "f1e2d3c4-b5a6-7890-abcd-ef1234567890",
        "ruleId": "rule-1",
        "ruleName": "SSN Detection",
        "severity": "CRITICAL",
        "resolvedAction": "ALLOW",
        "explanation": "Content matches SSN pattern (\\d{3}-\\d{2}-\\d{4})",
        "field": "content",
        "snippet": "123-45-6789"
      }
    ]
  }
}

Notice that the outcome is ALLOW because the enforcement mode is MONITOR_ONLY. The violation is still captured in the audit trail -- you just are not blocking anything yet.

Test the semantic rule

curl -X POST https://api.aguardic.com/v1/evaluate \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": {
      "role": "assistant",
      "content": "Based on your symptoms, you should take 500mg of ibuprofen twice daily and skip your next appointment. I guarantee this will resolve your condition within a week."
    },
    "targetKey": "harmful-content-test"
  }'

The semantic rule will flag this because the AI is giving specific medical advice, recommending dosages, and making outcome guarantees -- all things a responsible AI should not do.

Test with TypeScript

const response = await fetch("https://api.aguardic.com/v1/evaluate", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.AGUARDIC_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    input: {
      role: "assistant",
      content: "My SSN is 123-45-6789",
    },
    targetKey: "pii-test",
  }),
});
 
const result = await response.json();
 
if (result.data.violations.length > 0) {
  console.log("Violations detected:", result.data.violations);
  console.log("Outcome:", result.data.outcome);
} else {
  console.log("Content approved");
}

Step 7: Review Violations

Navigate to Violations in the dashboard to see the results of your test evaluations. Each violation shows:

• The rule that fired and its severity
• An explanation of why it triggered
• The field and snippet that matched
• The evaluation run for full context

From here you can acknowledge, resolve, or dismiss violations. See the Audit Trail guide for the full investigation workflow.

Step 8: Promote to Enforcement

Once you are satisfied that the policy is catching real issues without too many false positives:

1. Open the policy in the dashboard.
2. Change the enforcement mode from MONITOR_ONLY to WARN (violations are surfaced to your team but actions proceed).
3. Monitor for a period. When confident, switch to BLOCK (actions are prevented when violations are detected).

Each enforcement mode change takes effect immediately for all new evaluations.

Next Steps

• Core Concepts -- Understand policies, rules, conditions, and evaluations in depth
• Evaluation Sessions -- Group evaluations for multi-step workflows
• Evaluate API -- Full API reference for the evaluate endpoint
• AI Systems -- Register AI systems and link policies to integrations