How To Vet An AI Automation Partner: Five Questions To Ask Any Provider
Five questions that separate an AI provider with delivered results from one with a pitch deck, and the answers we give to each one with our own numbers.
To vet an AI automation partner, ask five questions and judge the answers on evidence, not enthusiasm: show me a delivered install, what runs without me, what it costs to keep running, who owns the build, and what happens when it breaks. A provider with real work behind it answers all five with names, numbers and specifics. A provider without it answers with a demo.
Owners are being pitched AI from every direction, and the reaction in Australian business forums is consistent: show some numbers. That is the right instinct. Below are the five questions, what a good answer sounds like, and how we answer each one ourselves.
The Bottom Line
- Ask for a delivered install with a named client and the client’s own figures.
- Ask what runs without you, and what still waits for a person.
- Ask what it costs to keep running after the build is paid for.
- Ask who owns the code, the accounts and the credentials at the end.
- Ask what happens when it breaks, and for an example of when it did.
Why Vetting Matters More Now
Most AI pitches look the same from the outside: a slick demo, a list of tools, a promise of hours saved. The demo proves the model can do something once. It does not prove the provider can make it run inside your business, every week, with your data and your staff.
The gap shows up in the research too. McKinsey’s November 2025 survey found 88% of organisations report regular AI use in at least one business function, yet only 39% attribute any EBIT impact to AI (McKinsey & Company, The State of AI, November 2025). It is a global survey, but the lesson travels. Buying AI is easy. Getting a result from it is the hard part, and that is what you are paying a partner for.
Question 1: Show Me A Delivered Install
Ask to see work that is finished and in use, at a business that will put its name to it. Not a mock-up, not a prototype, not a logo wall.
A good answer names the client, says what was built, and gives a before-and-after in the client’s own words, with the time period attached.
A weak answer is a case study with no names, a figure with no baseline, or “we can’t share client work”. Confidentiality is real, but a provider with nothing they can show has nothing you can check.
How we answer it. Three published case studies, each with the client’s own figures:
- Pritesh Hirani, four businesses: automated tasks went from about 3% to 25% in three weeks, with around seven hours a week back. His own numbers. In his words: “From this I’m already saving approximately $3,000 a month.”
- Drapery Co, three retail brands: about 65% of recurring work automated, up from almost none, and 22 to 30 hours a week back after six weeks, by Fernanda’s estimate. In her words: “I would have needed about three more staff to keep up.”
- Smashed Avo Festival: around 12 council-grade submission documents produced in-house over nine weeks, work Arthur previously outsourced at $3,000 to $5,000 a document.
These are different businesses measured in different ways, so we do not average them. You can read each one in full: Pritesh Hirani, Drapery Co and Smashed Avo Festival.
Question 2: What Runs Without Me?
This is the question that exposes a chatbot dressed up as automation. If every useful thing still needs someone to open a window and type, the hours do not come back.
A good answer lists the specific jobs that run on their own, and is just as clear about which actions wait for a person and why.
A weak answer is “it’s fully autonomous” or “the AI handles everything”. Nothing that touches money or clients should be fully autonomous on day one.
How we answer it. At Drapery Co, every email across six inboxes is sorted as it lands with a reply drafted, supplier invoices are drafted into Xero from the PDF, and a supplier invoice becomes a drafted Shopify product. A morning brief arrives before anything is opened. One morning it flagged a supplier order window closing that same day across two brands. Pritesh has six recurring jobs running on their own, including the invoice pipeline, inbox routing and a supplier-statement cross-check he had given up doing by hand.
What waits for a person: anything that moves money, reaches a client or publishes in the business’s name. Bills are drafted and held for sign-off. Replies are drafted, not sent. A task earns more freedom only after a run of clean approvals. We explain the design in human-in-the-loop automation.
Question 3: What Are The Running Costs, And Who Gets Paid?
The build price is only part of the cost. Ask what you pay every month after it is finished, and to whom.
A good answer separates the build from the running costs, names the tool subscriptions you will hold in your own name, and says plainly whether any ongoing fee goes to the provider.
A weak answer is a monthly platform fee that rivals the build, or a licence that stops the system working if you stop paying.
How we answer it. The engagement is fixed after the audit findings and priced on scope. No licence or subscription to us is attached to what we build. The running costs are the tools the business already pays for, plus any model or software accounts held in the business’s own name. Ongoing support after handover is optional and separately agreed, not a condition of the system working. For a fuller breakdown of what reasonable upkeep looks like, see what AI automation costs in Australia.
Question 4: Who Owns The Build?
Ask who holds the code, the accounts, the credentials and the documentation at the end of the engagement. Then ask what happens if you part ways.
A good answer is that you own all of it, and the handover is part of the scope, not an extra.
A weak answer is a system that lives in the provider’s account, on their platform, behind their login. That is renting, and the rent tends to go up.
How we answer it. At handover, the infrastructure, documentation, accounts and credentials transfer to the business, along with a full copy of the code. The systems run under accounts the business owns from the start. Closeout requires the agreed builds to be live and the handover verified from the client’s side.
Question 5: What Happens When It Breaks?
Everything breaks eventually. Passwords change, a platform updates its API, a laptop goes to sleep at the wrong time. Ask how the provider finds out, who fixes it, and for an example of a time it went wrong.
A good answer includes a real failure and how it was fixed. A provider who says nothing has ever broken has either not delivered much or is not being straight with you.
A weak answer is “it won’t break” or a support plan with no response terms.
How we answer it. On one install, the daily brief did not run reliably for about three weeks, because of the computer’s power settings, how security keys were stored and credentials syncing across two machines. A repair session fixed it, and that experience changed how we set up scheduled jobs and hand them over. On another, a point-of-sale platform’s access limits blocked two contracted automations, and only the core labour report went live later. We say this plainly because the next client’s system is built with those lessons in it.
Approval points also limit the damage when something does go wrong. A drafted bill that is wrong gets caught at sign-off. On one supplier invoice pipeline, any edit or void a person makes sends that supplier back to draft-only until it earns trust again.
Red Flags Worth Walking Away From
- No named client and no client’s own figures.
- A demo of a generic chatbot presented as automation of your business.
- A monthly fee you must keep paying for the system to keep working.
- The code, accounts or credentials staying with the provider.
- Claims of full autonomy on anything touching money or clients.
- A price quoted before anyone has looked at how your business runs.
That last one matters. A fixed quote before an audit is either padded for the unknowns or will grow once the work starts. The right order is to map the work first, rank it by hours and risk, and price what the evidence justifies. We cover more of the buying process in how to choose an AI automation agency in Australia.
What Should You Ask An AI Consultant Before You Sign?
Ask questions that leave a written answer you can hold the consultant to:
- Proof: Which delivered result is closest to this scope, what was measured and whose figure is it?
- Ownership: Who owns the workflow, accounts, code, documentation and data when the engagement ends?
- Access: Which systems and permissions are required, and how are they removed or transferred at handover?
- Approvals: Which actions wait for a person, and who can change those rules?
- Maintenance: Who handles failures, vendor changes and updates after handover?
- Measurement: What is the baseline, what will change, and how will both sides verify the result?
Good advice should also say what not to automate. Our AI consulting in Perth page explains how Echelon turns those answers into a ranked roadmap, while hiring an AI consultant in WA explains what a useful diagnostic should leave behind.
Buyers in Paddington, Sydney or Brisbane can apply the same checks with our Paddington AI consultant guide.
Frequently Asked Questions
How Do I Know If An AI Provider’s Results Are Real?
Ask for a named client, the client’s own before-and-after figures and the time period, then read or ask for the full case study. Results with no name, no baseline and no time frame cannot be checked. Self-reported figures are fine if they are labelled as the client’s own.
Should An AI Provider Charge A Monthly Fee?
Only for something you choose to keep buying, such as optional support. A fee the system needs in order to keep working means you are renting it. Running costs for tools and model accounts should sit in your business’s name.
What Should I Own At The End Of An AI Project?
The code, the documentation, the accounts the systems run under and the credentials. If the provider disappeared, your business should still be able to run, change and maintain what was built.
Is It A Bad Sign If A Provider Will Not Quote Upfront?
Not necessarily. A fixed quote before anyone has mapped your work is a guess. A paid audit or diagnostic that ends in a priced roadmap is a sounder basis, as long as the cost of that first step is clear.
What Is The Biggest Red Flag In An AI Pitch?
A demo with no delivered result behind it. If the provider cannot show finished work at a business that will put its name to it, the demo is all you are buying.
Is AI Consulting A Scam?
Not as a category, but the field is easy to enter, and plenty of people sell confidence instead of delivered work. The tell is evidence. A real consultant can show finished work at a named business with that client’s own figures, explain what still waits for a person, and tell you about a time something broke. If all you get is a demo and enthusiasm, you have your answer.
How Can I Tell If An AI Consultant Actually Knows AI?
Ask them to walk you through a failure. Someone who has built and run real systems can tell you exactly what broke on a past job, why, and what they changed afterwards, whether that was an expired credential, an API change or an edge case nobody mapped. Ask what they’d keep in draft-and-approve and why. Bluffers stay at the level of tools and buzzwords. Builders talk about specific systems, data and trade-offs.
Related Services
- AI Automation Agency Australia →
AI automation for established Australian businesses, built inside the systems they already run.
- AI Engineering →
Production AI systems built for a problem the business already knows it has, then handed over.
Sam co-founded Echelon AI Solutions and leads transformation strategy, client engagements and growth. He has built and operated businesses across marketing and AI education, and has guided companies in retail, trades, hospitality and professional services through operational change. His focus is making AI earn its place through measurable business performance.
More In Build, Hire Or DIY
See all Build, Hire Or DIY →
Build Vs Hire Vs DIY: The Honest AI Automation Call
Build in-house, hire an agency, or DIY no-code AI automation? An honest comparison of cost, speed, risk and ownership for a $1M+ Australian business.
Read itChatGPT Consultant: What They Do And When You Need One
A ChatGPT consultant turns a useful prompt into a controlled business workflow. Learn what the role covers, when to hire help, when to build in house and how Echelon AI Solutions approaches ChatGPT and OpenAI inside real operations.
Read itMachine Learning Consulting Firms: What They Do And How To Choose
Machine learning consulting firms design, train and operate models for prediction, classification and optimisation. Learn when custom machine learning is justified, when workflow automation with existing AI is the better route, how to compare firms and where Echelon AI Solutions fits.
Read it