Those approaches can work together. A model might extract a reason from a message while a fixed rule checks eligibility and a person approves an exception. The useful unit is the completed workflow, not the sophistication of one component.
Name the outcome and the decision
A fictional online store wants customers with eligible damaged orders to receive the correct resolution quickly. “Generate a fast reply” is only an intermediate task. The workflow also needs the order, delivery status, policy, evidence and a way to issue or decline the requested remedy.
List what must be known before each decision. If staff disagree about the policy, a model will not make the underlying rule consistent simply by writing fluent explanations. Resolve the policy or define the exception path first.
Match the technique to the task
| Task in the fictional workflow | Possible first approach | Main thing to test |
|---|---|---|
| Retrieve order status | API or existing system integration | Correct account and order |
| Apply a clear time limit | Explicit rule | Boundary dates and exceptions |
| Categorize a free-text complaint | AI-assisted classification | Misclassification on difficult cases |
| Approve an unusual remedy | Human review | Evidence, authority and consistency |
This is a design hypothesis, not a recommendation that every store should use these exact steps. A simpler intake form may eliminate ambiguity more cheaply than classification.
Count the review work
Suppose staff spend six minutes drafting each of 100 replies: 600 minutes. An illustrative AI-assisted workflow uses two minutes to review each reply and four minutes to correct each of 20 difficult cases. That totals 200 + 80 = 280 minutes of drafting-related work.
The apparent saving is 320 minutes, or 5 hours 20 minutes. It excludes integration, supervision, recurring tool cost and other resolution tasks. If incorrect replies create repeat contacts, the full customer outcome can still get worse.
Measure correct resolutions, repeat contacts and total staff effort. A sample should include ambiguous inputs and difficult exceptions, not just cases already easy to automate. Choose the evaluation period and failure limits before seeing the results.
Design the stop and escalation path
Name who can intervene, what evidence they need and how a mistaken action will be corrected. Keep a way to handle requests when the model or integration is unavailable. If an action is difficult to reverse, the approval and testing requirements deserve particular attention.
NIST's risk-management materials provide a framework for defining context, measuring performance and managing AI risk. They do not certify this fictional design or provide a universal acceptable error rate.
Run a small comparison
Map the process with the workflow guide. Identify its limiting step using the bottleneck resource. Compare a manual baseline, a simple rule-based change and an AI-assisted option where the task genuinely calls for it.
Use the experiment guide to document the intended benefit and the failure conditions. Keep the approach that improves the complete outcome at an acceptable cost and level of risk.
Sources and limits
Primary frameworks: NIST AI Risk Management Framework and NIST Generative AI Profile. All store tasks, counts and time estimates are original fictional HBW examples; they are not measured evidence of an AI productivity gain.