B2Shift · Published · Updated

Automation proposals are hard to compare because the underlying work varies enormously behind similar-sounding language. The questions below surface that variance before you sign anything.

Ask what they will actually build, not what a prepared example shows

A polished preview can show a model producing an appealing output. It does not prove the system can read your CRM, write back to it safely, and recover when an API call fails. Ask specifically which systems the proposal connects to, with what permissions, and what happens on a failed step — see AI agent vs chatbot vs workflow automation for the vocabulary this question depends on.

Ask for a measurable scope and its explicit assumptions

A quote without a defined success metric and a list of assumptions has simply moved the risk onto you. Ask what happens if the process turns out more exception-heavy than described, or if an integration's API has undocumented rate limits — a serious partner has an answer, because they have hit this before.

Ask who owns escalations and how mistakes are found

Every automation will eventually be wrong about something. Ask who is notified, how quickly, and what the audit trail looks like when a customer disputes an automated decision. An agency that has not thought about this has usually not built anything that survived contact with real customers.

Ask about data residency, access and what happens after the contract ends

Where does your data go, is it used to train anyone's models, and do you own the workflow logic if you switch providers later? Vendor lock-in is not inherently wrong, but it should be a decision you made, not one you discovered.

Ask for a reference for a comparable, not just any, project

A reference for a chatbot tells you little about a vendor's ability to build a workflow that writes to your ERP. Ask for a reference in the same category of complexity as what you are buying, and ask that reference specifically what went wrong and how it was handled — the answer to the second question matters more than the first.

Compare proposals with the same scorecard

Give every provider the same workflow description, volume assumptions and exception examples. Score the written response rather than relying on a polished preview. A low score on ownership or access can outweigh a high score on delivery speed because the buyer remains responsible for the process after launch.

Questions for evaluating an AI automation agency proposal
AreaAsk the providerEvidence to request
ScopeWhich systems, cases, exceptions and environments are included?Workflow diagram, assumptions and exclusions
OwnershipWho owns the workflow, data, credentials and production changes?Named roles and handover checklist
AccessWhat access is needed and how is it restricted?Permission list, credential handling and retention
SupportWho monitors failures and what response is included?Alert path, response window and support boundary
ExitHow can we take over or move the workflow?Export format, documentation and access revocation steps

Use a short, comparable proposal request

Send each candidate the same one-page brief: trigger, systems involved, monthly case volume, required outcome, known exceptions, data sensitivity, approval points and current handling time. Ask for a fixed discovery deliverable, a bounded implementation estimate, ongoing operating costs, acceptance criteria and explicit exclusions. Require the provider to show how a failed run is detected, retried safely and handed to a person with enough context to resolve it.

  • Who can change a live workflow, and how are changes reviewed?
  • Can we export workflow definitions, logs and business data without the supplier?
  • Which third-party subscriptions or usage fees are billed outside the proposal?
  • What must our team own each week after handover?

Define the decision before inviting agencies to pitch

Write down the business outcome, current process, eligible case volume, baseline effort, known failure costs and the person who will own the workflow. Decide what is out of scope and which actions require a human decision. This prevents a vendor from defining success as a convincing presentation when the buyer needs a reliable process that staff can operate. If the team cannot agree on the problem or data source, use discovery to resolve that before comparing implementation promises.

Give each candidate the same anonymised examples: a normal case, missing information, a duplicate and a case that must stop for review. Ask the agency to explain how it would handle each one and what evidence it would deliver at the end of discovery. A provider should be able to state what cannot yet be estimated and what access or decisions are needed from the client.

Evaluate the working method, not only a prepared example

A polished preview often uses prepared data and a narrow success path. Ask to see how the team tests a failure, inspects a system write, handles an expired credential and rolls back a change. If a live walkthrough would expose another client's information, accept a synthetic example and ask for a walk-through of the test artifacts rather than requesting confidential access. The useful evidence is a clear design, reproducible acceptance cases and an explanation of what happens when assumptions fail.

Check whether the people who will build the workflow can explain its design and whether the people who will support it are involved before launch. Ask who writes the runbook, maintains integrations, approves production changes and trains the client's owner. The answer should name roles and deliverables instead of relying on a general promise that the agency will 'take care of it.'

Score proposals against comparable evidence

Use a simple scorecard with a written definition for each rating. For example, score process fit, exception handling, data permissions, acceptance tests, handover, ongoing support and exit on the same scale for every supplier. Require a short evidence note for each score. If a proposal leaves a category unanswered, record it as unknown and request clarification rather than assuming the answer is favorable.

Weight the categories that matter to the workflow before reviewing bids. A low-risk internal summary may put more weight on integration fit and adoption; a workflow that updates financial or customer records may give greater weight to permissions, verification and recovery. Avoid allowing one impressive feature or the lowest price to compensate for a critical gap in security or operational ownership. Keep the final decision and unresolved assumptions in the procurement record.

Agree commercial boundaries, data rights and exit

Read the statement of work for included systems, volume assumptions, environments, change requests, acceptance steps, payment milestones and dependencies on the client's team. Confirm which third-party services are billed separately and who pays for usage, hosting, monitoring and support after launch. Avoid a quote that presents a fixed total while leaving the required connector, data cleanup or exception workflow undefined.

Agree who controls source data, workflow definitions, prompts, configuration, logs and credentials, and how each is transferred at handover. Define what the supplier may access during support, how that access is approved and revoked, and what happens to retained copies when work ends. Ask for a practical transition plan that another qualified operator can follow. The organisation's legal and procurement owners should review contract terms and data obligations for its circumstances.

Recognise warning signs during vendor selection

Be cautious when a provider guarantees a result before seeing the process and data, cannot explain a manual fallback, treats a successful preview as acceptance or avoids naming who owns failures after launch. Other warning signs include broad administrator access without a reason, hidden third-party charges, no change log, no way to export the workflow, and pressure to automate decisions that the business has not approved.

Ask for references or examples only where the provider has permission to share them, and distinguish illustrative workflow examples from customer deployments. A clear answer may include a limitation, a dependency or a recommendation to narrow the scope. That is more useful than an unsupported claim that every integration is easy or that human review will never be needed.

Run a structured interview and verify claims

Ask each shortlisted agency to walk through the same case from intake to final outcome, including one missing-data exception and one failed integration. Listen for how it identifies the authoritative system, validates model output, prevents duplicate writes, informs a reviewer and recovers after a timeout. Ask what would make the agency recommend a smaller scope or no automation at all. The answer reveals whether the team is evaluating process fit or simply trying to sell a preferred architecture.

Where a reference is available, ask the client about delivery clarity, change requests, handover quality, incident response and how much internal effort was required after launch. Do not ask for confidential documents or another client's data. A useful reference can confirm whether the described working practices were followed, while the proposal and contract should still state the terms for your own project. Record which claims are verified, which are illustrative and which remain assumptions before making a selection. Summarise the chosen scope, key evidence, unresolved risks and ownership decision so the approval can be understood later.

Sources

OpenAI — A practical guide to building agents

Discuss your workflow