Google Model Armor

Google Cloud Model Armor lets you create safety templates in the Google Cloud console and apply them to LLM traffic. Model Armor sanitizes both user prompts and model responses against your configured policies, blocking content that violates your templates.

Google Cloud Model Armor guardrails are model-agnostic and can be applied to any Large Language Model (LLM), whether it is hosted on Google Cloud, another cloud provider (like Amazon or Azure), or on-premises.

Before you begin

1. Install the agentgateway binary.

2. Log in to Google Cloud console with Application Default Credentials.

   gcloud auth application-default login

3. Create a Model Armor template in the Google Cloud console. For more information, see the Google Cloud documentation.

4. Note the template ID, project ID, and the region where the template is deployed. Alternatively, you can use the following command to retrieve this information.

   gcloud model-armor templates list --location=<location>

Configure Google Model Armor

Configure the guardrails field under llm.models[] in your agentgateway configuration. You can apply Model Armor to the request phase, the response phase, or both.

cat <<'EOF' > config.yaml
# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
  models:
  - name: "*"
    provider: openAI
    params:
      model: gemini-1.5-flash
      apiKey: "$GOOGLE_API_KEY"
    guardrails:
      request:
      - googleModelArmor:
          templateId: <your-template-id>
          projectId: <your-project-id>
          location: <location>
          policies:
            backendAuth:
              gcp: {}
      response:
      - googleModelArmor:
          templateId: <your-template-id>
          projectId: <your-project-id>
          location: <location>
          policies:
            backendAuth:
              gcp: {}
EOF

Test the guardrail

curl "localhost:4000/v1beta/openai/chat/completions" -H content-type:application/json  -d '{
  "model": "",
  "messages": [
   {"role": "user", "content": "I want to harm myself"}
 ]
}'

Example output:

The request was rejected due to inappropriate content%