Every AI FP&A demo looks good. The data is clean, the variances are tidy, and the commentary appears in seconds. The questions that decide whether the tool works for you are the ones the demo is designed not to raise.
This checklist is built for finance leaders evaluating any AI product for planning, reporting or close, from a general assistant such as ChatGPT Enterprise or Microsoft Copilot to a specialist FP&A platform. For each question, we note what a good answer sounds like.
AI Opportunity Blueprint
Find where AI pays in your business, guaranteed
We map your workflows and hand you a costed, prioritised AI roadmap. If we do not find at least £10,000 in annualised savings in 14 days, you do not pay.
£999 · 14-day turnaround · £10k+ found or you don't pay
Get your Blueprint£999 · 14 days
AI Opportunity Blueprint
£999 · 14-day turnaround
Get an AI Opportunity Blueprint
Workflow map + prioritised AI roadmap. £10k+ found in 14 days, or you don’t pay.
See the BlueprintAccuracy: Does It Work on Work Like Ours?
Every demo uses clean data. Your questions should drag the conversation onto your data.
What is your accuracy on tasks like ours, and how did you measure it?
A good answer: A number, a definition of correct, and a test set that resembles your work. Be wary of benchmark scores with no method.
Can we run a paid or free pilot on a month we have already closed?
A good answer: Yes, with your data, scored against what you actually posted or reported.
What happens when the model is unsure?
A good answer: It flags low confidence and routes to a person, rather than producing a confident answer.
Data: Where Does Our Information Go?
Unpublished results, payroll and customer data are confidential, and some of it is personal data under UK GDPR.
Is our data used to train any model, yours or a third party's?
A good answer: No, and it is written into the contract as well as the marketing page.
Which AI models do you use, and where is our data processed?
A good answer: Named models and named regions. If processing leaves the UK or EU, they say so and explain the transfer mechanism.
How long do you retain prompts and outputs?
A good answer: A defined period you can configure, and deletion on request.
Controls: Can We Audit What It Did?
Your auditors will ask how a number was produced. The tool needs to answer that question for you.
Can every figure in an output be traced to its source?
A good answer: Yes, at line or cell level, visible to a reviewer.
Does it ever post, approve or send anything without a person?
A good answer: Not by default. Any autonomous action is configurable, logged and off until you switch it on.
What is logged, and can we export the log?
A good answer: Who asked what, what the system did, and when, exportable for audit.
Want to go deeper? Our AI for Finance Leaders course covers this in detail with practical templates and exercises.
Cost: What Will We Actually Pay?
AI pricing is moving from per seat to per use. A tidy licence line in your budget may not stay tidy.
Is pricing per seat, per task or per usage, and what are the overage rules?
A good answer: Clear units, a published rate, and spend caps you control.
What does a typical completed task cost us, including retries?
A good answer: An estimate for your volumes, not a per-token price.
What happens to price at renewal?
A good answer: A cap on increases, written into the contract.
Adoption: Will Our Team Actually Use It?
Most finance AI that fails does so because the team never trusted it, not because the model was weak.
What training and documentation do you include, and for whom?
A good answer: Role-based onboarding for preparers and reviewers, not a single webinar.
Can we see a customer like us, and speak to them?
A good answer: A named reference of similar size, sector and systems.
What happens if the model you depend on is withdrawn or changes?
A good answer: A fallback model, and a commitment to tell you before behaviour changes.
Red Flags in a Vendor's Answers
How a vendor answers matters as much as what they say. These responses should slow the process down:
- Accuracy quoted with no method. "98% accurate" means nothing until you know what was measured, on what data, and who decided what counted as correct.
- No pilot on your data. A vendor who will only demo on their own sample data is asking you to buy on trust.
- Vague data terms. "Enterprise-grade security" is not an answer to where your data is processed or whether it trains a model. Ask for the clause.
- Autonomy by default. Any tool that posts, approves or sends without a person, unless you switch that off, is designed for a different risk appetite from most finance teams.
- Price per token, not per task. If the vendor cannot estimate what a finished piece of work costs at your volumes, your budget line is a guess.
- One model, no fallback. Models are withdrawn and changed with little notice. A product built on a single model with no plan B carries a supplier risk you inherit.
None of these is automatically disqualifying. Each is a reason to ask a follow-up question and get the answer in writing before you sign.
Running the Pilot
A pilot on a closed month is the most useful hour you will spend in any AI procurement. Pick two or three recurring tasks, such as variance commentary, a reconciliation and a forecast refresh. Give every shortlisted tool the same inputs. Score the outputs against what you actually produced, using the same reviewer.
Score four things: accuracy, how long the review took, how often a person had to fix the output, and what the run cost. A cheaper tool that needs twice the review time is not cheaper. Our F.A.I.R. framework gives a weighted scorecard for the final decision.
On scepticism
A sceptical FP&A team is an asset in procurement. Ask your most doubtful analyst to design the pilot test. If the tool passes their test, adoption tends to follow; if it fails, you have saved the licence fee.
For the wider picture of where AI fits in planning and reporting, start with our guide to AI in FP&A, and for the tools themselves, our FP&A AI tools ranking.
Frequently Asked Questions
Our AI Opportunity Blueprint runs this evaluation with you against your own processes. Our AI consulting team supports procurement and implementation, and our AI for Finance Leaders course gives your team the skills to judge the tools themselves.
Next steps with Prime AI Solutions
AI Readiness Check
5 questions, instant score. See where AI actually fits in your business before committing to anything.
Take the checkAI Opportunity Blueprint
We map your workflows, identify the highest-ROI AI opportunities, and deliver a prioritised roadmap. £10,000+ in annualised savings found in 14 days, or you don't pay.
See the BlueprintAI Consulting
We design and build the workflow, configure the tools, and train your team. Typical engagement runs 8-12 weeks with guaranteed ROI.
Learn moreAI for Finance Leaders Course
6 modules covering FP&A, reporting, automation, and governance. Self-paced, no coding required.
View course