Build an AI Support Agent That Knows When to Say No

OpenAI's newest AI employee already handles three out of every four support tickets it sees. Yours can too, and you do not need OpenAI's budget to build it.

It is a Tuesday night in Manchester. Owen runs Birchwood Home, a seven-person shop that ships side tables and reading lamps across the UK. A customer messages the support chat: my lamp arrived cracked, I want a full refund plus the return postage, tonight.

The chatbot Owen bought off a marketplace does not check anything. It sees the word refund. It sees a frustrated customer. It says yes.

Owen finds out three weeks later, when the numbers in his refund ledger stop adding up.

That is not a chatbot problem. That is a missing layer problem. A chatbot answers. An agent decides. Nobody ever told Owen's bot where the line was.

What is an AI agent policy layer, and why does your support bot need one?

A policy layer is a set of rules that sits between your AI model and your customer, checking every message before it goes out: what the agent is allowed to approve, what needs a human, and what it should never say. Without one, the model is the entire system. With one, the model is just the part that writes sentences, while the policy layer decides whether those sentences are allowed to ship.

Most small business chatbots skip this. They wire a model straight to a chat widget and call it done. It works fine for "what are your opening hours." It falls apart the moment a customer asks for a refund over your limit, brings up a competitor's price, or types in a card number by mistake. The model will happily keep going. It does not know it is not supposed to.

Why now: OpenAI just proved this at enterprise scale

On 22 July 2026, OpenAI launched Presence, a product for deploying AI agents that "can answer questions, resolve issues, use company systems, take approved actions, and escalate to people when needed." OpenAI now runs its own English-language phone support line on it.

Here is the catch: Presence is not for you. It is a limited, enterprise-only rollout, deployed with OpenAI's own engineers, not something a seven-person shop can sign up for. But the architecture behind it is not a secret. Sweta Kumari, who runs the YouTube channel Agentic Data Lab, rebuilt the same core system over a weekend using entirely free-tier tools and posted the walkthrough days after the launch. That is the version worth studying. Not because it is OpenAI's, but because it fits your budget.

Isn't this just a chatbot with extra steps?

No, and the difference matters more than the terminology. A chatbot is a single call: message in, reply out. An agent, built this way, is a small state machine. Every customer message moves through checkpoints, in order: identity check, intent read, policy check, draft a reply, guardrail scan, decide whether to escalate, then either send the reply or open a ticket. Every one of those steps can read what happened before it and change what happens next.

That routing is the entire point. If the policy check fails, the message never reaches a human-facing reply, it goes straight to a person. If the guardrail scan fails, the system does not ship a bad answer, it asks the model to try again. If confidence is low twice in a row, it escalates on its own. None of that is possible when a prompt is the whole system. A prompt is an instruction. It is not a safety mechanism, and it never was.

How to build a governed AI support agent for your small business

You do not need OpenAI's engineering team for this. The free-stack rebuild uses LangGraph to wire the steps together, Groq's free-tier Llama 3.3 70B as the model that actually talks to customers, and Claude Haiku, a few cents a day, as a second model that only grades conversations after they end. Here is the build, in order.

1. Give the agent a memory, not just a prompt. Before you write a single response rule, define a state object: session ID, the customer's message, what the model thinks the customer wants, whether a policy was violated, the confidence score, the escalation reason, if any, and the final reply. Every step in the pipeline reads and writes to this one object. Without it, you have a pile of disconnected function calls. With it, you have a workflow that remembers what already happened.
2. Write your policies as data, not code. Put your rules in a plain config file, not buried in the prompt: "never approve a refund over £75 without a human," "never mention named competitors," "never discuss pricing changes before they are announced." A refund threshold changes far more often than your code should. When the policy check reads a message and a rule fires, the conversation routes straight to a human, before the model ever gets to draft an answer.
3. Let the model draft, then check the output too. The policy layer filters what comes in. A separate guardrail layer checks what goes out, because a perfectly innocent question can still produce a risky answer. In the free-stack rebuild, the guardrail scans every drafted reply for personal data it should not be repeating, competitor names, hallucination-flavoured phrases like "I believe" or "probably," and unfinished sentences. Fail the scan twice and the conversation escalates automatically.
4. Set the four reasons a human takes over. Pick them in advance so nobody has to argue about it mid-ticket: a policy violation, a guardrail that keeps failing, confidence that stays low, or a customer who flatly asks for a person. Wire all four into one escalation check.
5. Hand off with the whole story, not a cold ticket. When the agent escalates, it should open a ticket that already contains the reason, a priority, and the full conversation so far. The point of automating support is not to dump confusion on a human faster. A member of staff picking up that ticket should never have to ask the customer to repeat themselves.
6. Grade every conversation after it ends. Once a session closes, send the transcript to a second, cheaper model and ask it three questions: was the issue actually resolved, did the agent follow policy, and how accurate was the answer. Store the score. This is how you move from "it feels like it's working" to a number you can track week over week.
7. Let a weekly job suggest fixes, but keep a human in charge of approving them. Once a week, pull the failures, the guardrail trips, and the low-scoring sessions, and ask a model to propose three specific policy or knowledge-base fixes. Do not let the agent apply its own suggestions. Read them, approve the ones that make sense, and only then update the policy file. That single approval step is what keeps a self-improving agent from quietly rewriting its own rules.

Put a simple dashboard on top of it, even a basic one, showing total sessions, how often it escalates, how many policy violations fired, and the average grading score. That dashboard is the difference between running a demo and running a system you can actually manage.

What a small business gets that a marketplace chatbot cannot give it

A £20-a-month chatbot answers questions. It does not know when to stop. The system above costs closer to a weekend of setup and a few dollars a month in API calls, mostly for the evaluation model, since the primary model runs on Groq's free tier. What you get in return is not a smarter bot. It is a bot that knows its own limits, logs its own mistakes, and gets a little more precise every week without you touching a single prompt by hand.

What you have after this

A week in, you have a support agent that stops itself before it approves a refund it should not, and hands off cleanly when it is out of its depth. A month in, you have a log of every policy trip and guardrail catch, which tells you exactly where your actual policies are unclear, not where you guessed they might be. Six months in, you have an agent whose policy file has been quietly tightened a dozen times by a human reading its own weekly report, and a support inbox that only gets the tickets that genuinely need a person.

Same team. Same ticket volume. A completely different machine underneath it.