How to plan an AI automation pilot for your business

A useful pilot starts with one job that people can describe and judge: extracting fields from a document, drafting a summary, routing an enquiry, or finding an answer in an approved knowledge base. Record the input, expected output, user, and action that follows. Avoid making the first objective a broad transformation of an entire department.

Check whether ordinary workflow automation can solve the problem. If every decision follows explicit rules and the inputs are structured, a conventional application may be simpler. AI becomes more relevant when language or varied documents are central to the task. The pilot should test that choice, not assume it.

Collect examples that reflect normal work and difficult exceptions. Include incomplete inputs, conflicting information, unusual formats, and cases where the correct response is to ask for help. Obtain permission to use the data and remove information that is unnecessary for the test.

Have someone who understands the task describe what a good result looks like. For extraction, check fields individually. For summaries, check missing facts and unsupported statements. For question answering, check whether the response is supported by the permitted source material. Keep a separate set of examples for evaluation so improvements are not judged only on familiar samples.

Decide what the system can read, what it can change, and who can see its output. A pilot that drafts a response has a different risk from one that sends it automatically. Start with human review where a mistake could materially affect a customer, financial record, or clinical workflow.

In Salt’s clinical intake project, recorded responses, timestamped transcripts, and structured summaries support clinician review. That is an example of AI assisting documentation within a broader application. It does not establish that an unrelated workflow can safely run without supervision.

Choose acceptance criteria before the demonstration. Include error types, reviewer effort, turnaround time, and operating cost. A fluent result may still require enough correction to erase the benefit. Compare the proposed workflow with the current process using similar examples.

Record failures instead of hiding them in an average. Missing a critical field may matter more than several formatting mistakes. Define which errors stop a release, which route the task to a person, and which are acceptable within a clearly communicated limitation.

The scope should include the workflow, approved data sources, evaluation examples, results, known failure modes, and a recommendation for the next step. Define access to the prototype and ownership of any custom code or configuration. Budget for integration and review work as well as model usage.

Agree on a decision point: stop, revise the task, or proceed to a limited rollout. A successful demonstration is not the same as production readiness. Wider use may require stronger permissions, monitoring, incident handling, and repeated evaluation when documents, prompts, or models change.