All articles

Blog

Build the Agent Around the Domain

A developer who knows the system is the fastest person to fix a customer issue. They understand the code, the design and where things usually break. What they don't know is the customer. What are they running? How do they use the product? What changed on their side? What do the logs show?

Support levels exist to answer those questions. But every hand-off adds waiting, and some detail is lost each time. The fastest fix happens when one person has both the technical understanding and the customer context. That person rarely exists.

We set out to build that person as an AI engineer. We assumed the hard part would be the AI. It wasn't. The hard part was the domain, and one decision inside it: which parts of the work must happen the same way every time, and which parts need judgment. This article is what we learned.

Hand-offs today vs. one review
Hand-offs today vs. one review

Access to everything, understanding of nothing

Early on, our agents could reach the code, the logs, the tickets and the docs. They looked impressive in a demo and were unreliable on real tickets. One answered about the wrong customer. One skipped the logs and guessed. One found the cause, then reasoned it away.

Each mistake was fixed in a different place. The wrong customer was fixed by the platform: the customer is pinned before the AI starts. The guessing was fixed by grounding: an answer without evidence is flagged. The reasoned-away evidence was fixed by review: a second agent challenges the reasoning. A smarter model fixed none of these. A model of the domain is what fixed them.

Three failures, three fixes
Three failures, three fixes

Lesson 1: Split the rules from the judgment

Support work has two kinds of steps. Some must happen the same way every time: which customer this is, what data the AI may see, who approves, what gets posted, when the task closes. Others can't be scripted: working out why this customer's invoices stopped syncing. No two tickets take the same path to a root cause. Diagnosis takes judgment, and judgment is what AI is good at.

So we split them. The platform owns the rules. The AI owns the judgment.

Platform rules around AI judgment
Platform rules around AI judgment

A prompt is a request. A platform rule is a guarantee. We define each AI engineer as a solution: where the work comes from, how the customer is identified, who does the work, what happens to the result and when the task is done. Everything in that definition runs as a rule, and every action is recorded and checked against permissions.

Lesson 2: Pin the customer first

In support, the first question is always the same: who is this customer and what do they run? Two customers on the same product rarely hit the same problem in the same way. They differ in configuration, data, how they use the product, and sometimes the release they run. An answer that is right for one can be wrong for the next.

So the customer is never the AI's call. The platform pins the customer and their setup before the AI starts, and everything the AI reads after that is this customer's: their code, their logs, their tickets. This is where most of our engineering effort went: mapping each customer to their setup, understanding the code they run, and keeping each customer's data separate from every other's. None of it shows up in a demo. It decides whether the answer is right.

The customer is pinned before the AI starts
The customer is pinned before the AI starts

Lesson 3: Give the judgment the roles it already has

For the judgment work, support already has a shape that works. Someone investigates. Someone more experienced challenges the reasoning before anything goes out. We copied that shape.

Think of the AI engineer as an external engineer working inside your organization: a clear role, access to only what the role needs, and a reviewer who checks the work. Inside, it is two agents:

Investigator and reviewer: job, tools, what each can change, and who each hands off to
Investigator and reviewer: job, tools, what each can change, and who each hands off to

The reviewer challenges the reasoning, not the rules. Do the cited lines of code say what the draft claims? Does the root cause explain every symptom the customer reported? Is any workaround safe and limited to this customer? It can send the draft back; it cannot take over. Its checklist can be your team's own review lessons.

Reviewed ticket diagnosis: 2 agents, 1 review, 1 trust gate
Reviewed ticket diagnosis: 2 agents, 1 review, 1 trust gate

Lesson 4: Earn trust from the record

Generic AI benchmarks told us little. What matters in support is whether the diagnosis rests on evidence and whether the team accepts it.

Trust is earned the way a support lead already judges a new hire: by a run of good work. In Shadow, the AI engineer works every ticket but nothing goes out. In Copilot, a support engineer approves before anything is posted. In Autopilot, it posts on its own. A run of accepted diagnoses moves it to Copilot, and a longer run moves it to Autopilot. Each team sets how long each run must be. One rejected diagnosis resets the streak, repeated misses move it back down, and you can take trust back at any time. We chose a streak over a weighted score because people trust a clean run more than a score.

Trust levels: up by streak, down on misses
Trust levels: up by streak, down on misses

An accepted diagnosis is not proof that it was right. A busy engineer can approve an answer that reads well, and a closed ticket is not always a solved one. So we are moving the measure past approval to what happened next: was a fix linked to the ticket, did it ship, and did the ticket stay closed? Trust and learning should rest on that evidence, not on a click.

Lesson 5: Put the review where the decision already happens

Our first version asked support engineers to come to us: drafts waited in our app for a review. Most never got one. People were busy, the ticket was in Jira, and our app was one more tab. A review that sits outside the team's flow does not get done, and an agent nobody reviews cannot earn trust.

So the AI engineer now works inside the flow the team already has. The ticket stays where it is, the review happens wherever the team already works (in the ticket in Jira, or in Slack), and the support levels stay. One diagnosis also goes out in a different shape for each reader: the support engineer gets the fix and the evidence, and the developer gets the code note. A chain of hand-offs becomes one review.

Here is an example of how a ticket flows. A customer reports that invoices stopped syncing over the weekend. The platform pins the customer and their setup. The investigator finds authentication errors in their logs starting Saturday morning, sees in their ticket history that their integration credential was due to expire, and finds in the code that the sync job fails quietly when a credential expires. The reviewer checks that this root cause explains the whole report and that no other cause fits the logs better. The support engineer gets the fix (renew the credential) with the evidence; the developer gets a note that the sync job should alert, not fail quietly.

One ticket: who does each step
One ticket: who does each step

That is the person we set out to build: technical understanding and customer context in one place. Support engineers and developers spend their time on judgment instead of searching. Customers get fewer questions sent back, a faster first answer, and an answer that fits their setup.

Where this goes

The lesson is not specific to support. Any work with clear rules, real evidence and a definition of done can be built the same way: put the rules in the platform, give the judgment to the AI, and review the judgment.

The same pattern carries past diagnosis. When a support engineer corrects an answer, the correction becomes knowledge the next ticket can use, and it stays when people move on. When a diagnosis names a fix, the AI engineer can draft it for a developer to approve.

If you are building agents, start with the domain, not the model. That is what we are doing at buildwright.ai.

More from the BuildWright team

Get started

Grade BuildWright on your own escalations

Read-only access, personal data masked, running in your own cloud. Start on the sample app today, or bring a ticket your team already solved and judge what it finds.

Not ready for a call? Send us a note