Benchmark

Solve the escalations that bounce between teams for days

Watch BuildWright trace a real customer-reported bug to the exact file and method the shipped fix changed, using nothing but the customer’s description.

Two engineers at a desk checking code together, one pointing at a line on the laptop screen

The problem · ERPNext

“The invoice totals never reconcile…”

Off by a few cents, only sometimes, and nothing in the ticket says where to look. This kind of ticket bounces between teams for days.

frappe/erpnext · Bug reportMar 30, 2026

“On some multi-item invoices, the item-wise tax breakup doesn’t add up to the total tax. It’s off by a few cents, only on certain rate and quantity combinations. The header total looks right; the per-item breakdown doesn’t reconcile.”

Fixed in ERPNext pull request #53925

The solution

BuildWright investigated

BuildWright traced the missing cents to lines 540 to 556 of the set_item_wise_tax() method in taxes_and_totals.py.

Tax was rounded on each line separately, so small differences drifted from the header total. ERPNext pull request #53925 changed the lines BuildWright named.

BuildWright diagnosis
File
erpnext/controllers/taxes_and_totals.py
Method
set_item_wise_tax()
Lines
540–556
Cause
Tax rounded on each line separately. Small differences add up and drift from the header total.

Matched the shipped fix. ERPNext PR #53925 changed the same lines.

Technical detail

It works the way your team would

A read-only index of the code, related knowledge, and a grade on the evidence. The only input was the customer’s description.

With its knowledge layer switched off, BuildWright’s production agent matches the reference agent on τ²-bench, Sierra Research’s public benchmark for AI support agents.

τ²-bench domainBuildWrightReference agent
Retail0.820.81–0.83
Airline0.790.80

Resolution rate across 50 tasks per domain, graded automatically.

Results

BuildWright doesn’t mark its own homework

SUP-214Ungrounded

BuildWright could not support a diagnosis

Not enough evidence to name a root cause

no indexed code repository

Connect the repository for this service, or add the error log from the time of the report, and I will investigate again.

A tool that says “I don't know”

In one run it reported “no indexed code repository” instead of inventing a file path.

SUP-302High grounding

BuildWright used a past resolution

Same rounding issue, different customer

From SUP-128 · resolvedPer-line rounding in set_item_wise_tax()

Learned resolutions surface the facts it needs roughly 3x more reliably the next time.

Faster with every fix

Once it learns a resolution, the facts it needs surface roughly 3x more reliably the next time.

Trust level · BillingRead-only
  • ShadowDrafts only. Nothing posts.
  • CopilotA person approves every post.
  • AutopilotOnly for proven case types.

Read-only by design

It never changes code, data or production. Your team sets the trust level.

Get started

Grade BuildWright on your own escalations

Read-only access, personal data masked, running in your own cloud. Start on the sample app today, or bring a ticket your team already solved and judge what it finds.

Not ready for a call? Send us a note