95% accuracy. A very expensive 5%.

A single accuracy percentage is too small a description of an important business system. It leaves out which cases failed, what happened next and how much effort people spent correcting the result. The 95% in this headline is an illustrative number, not a measured result from my clients or a claim about a particular model.

Picture a fictional assistant handling one hundred requests. Ninety-five answers satisfy a simple check. The other five include a harmless wording problem, a missed urgent request and a confidential answer shown to the wrong person. Counting each as one equal error produces arithmetic. It does not produce a useful operating decision.

Define the accepted outcome before the score

Write down what the system is supposed to help someone complete. For a quotation workflow, success might require the correct customer, current commercial terms and approval from the right person. An answer can be fluent and factually plausible while failing that whole task.

Make the acceptance criteria observable. If two qualified reviewers disagree, examine the instruction and business policy before blaming the model. Preserve genuine ambiguity in the test rather than quietly relabelling it until the chart looks better. Sometimes the correct answer is a request for clarification.

Split errors by consequence

NIST’s Generative AI Profile is a voluntary resource for incorporating trustworthiness into AI design, use and evaluation. The practical scorecard below is my own way to structure a business conversation, not a NIST scoring formula or certification scheme.

Set non-negotiable release conditions separately from averages. For example, a team might require a permission test to pass before a feature reaches any users. Other defects may be acceptable within a supervised pilot. The decision belongs with the responsible business and technical owners, who need to understand the limits of the evidence.

Keep the awkward cases in the room

Build test groups around work that differs meaningfully: language, document condition, customer type, missing information or request complexity. Report sample size and results for each group. A large set of easy cases can hide a small group that consistently receives bad service.

Do not mistake a small sample with no observed failures for proof of zero risk. Keep the scope of the result visible. If you tested clean English invoices from one supplier, that is what you tested. The product has not earned a claim about every document in every market.

Maintain both a fixed regression set and fresh cases collected through an approved process. Protect holdout examples from training and routine prompt tweaking. Otherwise the team can gradually memorise the exam while congratulating itself on improved intelligence.

Use model judges with an appeal route

An automated judge can help organise review, but its output is another system output to validate. Compare it with qualified human judgements on relevant cases. Check disagreements, sensitivity to phrasing and whether it rewards confident verbosity. Do not let the same untested assumption generate answers, grade them and declare the project complete.

Where an objective check exists, use it. A required field can be checked for presence; a calculation can be recomputed; a permitted action can be checked against an explicit rule. Other qualities require contextual judgement. Keep those different forms of evidence distinguishable.

Test changes as changes

A model-card result can inform investigation, but it is not your application test. Google DeepMind’s model cards are an example of supplier-level documentation to read alongside your own evaluation. Your workflow adds prompts, sources, tools, permissions and users that the supplier’s report may not cover.

When any of those parts changes, compare the old and new setup on the same relevant cases. Inspect regressions even when the overall result improves. Record the versions, decision, known weaknesses and rollback or fallback approach. Someone should be able to explain why the release was approved after the enthusiastic meeting has ended.

Make the result useful to management

Summarise what the system can do, where it needs supervision and which failures still block expansion. Give the team a decision, such as continue a bounded pilot, fix retrieval, narrow the supported workflow or stop. “The model is better” is rarely enough to approve a wider deployment.

The same discipline helps decide between fine-tuning and retrieval and prevents a pilot with no clear business value from surviving indefinitely. I help teams connect technical evidence to those product decisions, with a scorecard that can survive a customer question as well as a demo.

Questions people ask

Is 95% AI accuracy good enough for business use?

It depends on the task, sample, severity of failures and controls. The headline’s 95% is illustrative. A single percentage cannot establish suitability for a specific workflow.

What should a business measure besides AI accuracy?

Measure task completion, error severity, supported coverage, human correction effort, latency and cost per accepted outcome. Agree release-blocking conditions separately.

Can another AI model judge all the answers?

An automated judge can assist evaluation, but validate it against relevant human judgements and objective checks. Keep a way to inspect disagreements and challenge the score.