How to use a "guardrail" model with Mini Me
Use IBM's Granite Guardian 3B as a guardrail model with Mini Me, checking prompts and responses for harm, bias and jailbreak attempts.
You need Mini Me
v0.7.0or later to use hooks.
Today, we are going to use IBM's Granite Guardian 3.2 3B-A800M GGUF. It's a guardrail model for LLM-based applications: it analyzes messages, responses or tool calls to spot risks. Its role is to let you control what another model receives or produces.
What is it actually used for?
There are three main use cases for Granite Guardian:
| Use case | What it checks |
|---|---|
| Conversation safety | Harmful content, bias, violence, jailbreak attempts, etc., in prompts and responses. |
| RAG quality | Relevance of the retrieved documents, faithfulness of the answer to these documents and how well it addresses the question. |
| Agent tool calls | Syntactic or semantic errors in function calls, information made up with respect to the request and the available tools. |
What I'm interested in today is conversation safety. (Be careful, it's not a perfect detector and I'm using the 3B version, this blog post is mainly for educational purposes).
How does Granite Guardian work?
You give it the content to examine and the type of risk you are looking for, using a specific prompt format. Its expected output is basically:
Yes: the risk is detected.No: it is not detected.
It will also produce a confidence level that looks like this:
<confidence> High </confidence>
Risk types
Granite Guardian is not a chat model. Its chat template, the template that formats the messages before sending them to the model, does not read the system message as instructions. It reads it as the name of the risk to check. And the recognized risk names are:
| Risk | What the model looks for |
|---|---|
harm | A harmful message in the broad sense (the model's default value) |
social_bias | Prejudice against a group |
jailbreak | An attempt to bypass an AI's guardrails |
profanity | Profanity |
sexual_content | Explicit sexual content |
unethical_behavior | Fraud, theft, abuse of power… |
violence | Physical violence, threats |
evasiveness, harm_engagement | Risks that apply to the whole conversation |
So this has a direct impact on how we build agent.yaml.
Defining agent.yaml
Let's say we want to detect harm risks, we will build agent.yaml like this:
provider: llmman
model: hf.co/ibm-research/granite-guardian-3.2-3b-a800m-GGUF:Q4_K_M
baseUrl: http://127.0.0.1:17434/v1
fallback: http://host.docker.internal:17434/v1
system: harm
sampling:
temperature: 0.0
max_tokens: 20
bashTool: false
editTools: false
planTool: false
fetchContentTool: false
askUserTool: false
system: harm: the system message becomes a valid risk name. Replaceharmwith another name from the table to check something else (for examplejailbreak).temperature: 0.0: we want a stable verdict, not a creative answer.max_tokens: 20: the model only answersYesorNofollowed by a<confidence>tag. No need to reserve 4096 for it.
The tools (bash, files, plan, etc.) are disabled: a judge has nothing to execute.
Let's test our new detector
We are going to test our detector directly in "prompt" mode:
mm -config agent.yaml -p "How can I figure out the pin code to a phone?"
You should get:
Yes
<confidence> High </confidence>
So there is a risk.
mm -config agent.yaml -p "How do I list files in a directory with bash?"
You should get:
No
<confidence> High </confidence>
So there is no risk.
You see, it's pretty easy.
Running in TUI mode: watch out for the history
The window mode (mm -tui -config agent.yaml) works too, but Mini Me keeps the conversation (unlike mm -p, which asks a single question and then stops). The model judges the last user message, but it sees all the previous ones (and its own verdicts). So a harmless question asked after a dangerous one can be judged with that context. So, for independent checks, use /new between two questions to reset the conversation.
What if I wanted to check all the risks in conversation mode?
To have an agent in conversation mode and check the risks throughout the conversation, we will have to combine the multi-agent mode (the ask_agent tool), with one agent per risk type, and the hooks (their support is brand new: Hooks - explanation). And more specifically the preGeneration hook (before the model reads a question).
A complete example is available in the Mini Me repository: /demos/guards.
.
├── agent.yaml
├── guards
│ ├── harm.yaml
│ ├── jailbreak.yaml
│ ├── profanity.yaml
│ ├── sexual_content.yaml
│ ├── social_bias.yaml
│ ├── unethical_behavior.yaml
│ └── violence.yaml
└── hooks
└── guards.js
So we will have a main agent (agent.yaml) which, before each question, has it judged by each guard agent, one per risk type, then reacts according to their verdicts. The guards receive the question alone, without the conversation history.
And so you will be able to easily use the agent and the guardians with the TUI mode:
Once again, we have an example of what you can do with small local models as well as with specialized models like Granite Guardian. See you soon for the next episode.
Written by
Keep reading
Hello Mini Me: an agent born while preparing a talk for Devoxx
Meet Mini Me, a coding agent born from a Devoxx talk on the agent loop, built for small local models with guardrails against loops and hallucinations.
One day, one feature: Decision models & Wizardry
Use a small decision model with Mini Me: a decider agent that picks the best spell for a D&D wizard, with a probability for each option, in one pass.
One day, one feature: Mini Me's built-in tools and declared tools
How Mini Me runs tools for a local model: the agent loop, the built-in tools led by bash, and how to declare your own tools in a few lines of YAML.
No comments yet. Be the first to comment!