How to trust your AI-assisted data analysis
AI tools can easily produce data analysis that looks authoritative without being verifiable. For anyone using AI to analyse data, whether you are the person doing the work or the person relying on the output, this post highlights what you should be thinking about.
Do those numbers look right?
Imagine an analysis lands on your desk. It has specific numbers, a clear narrative, a well-formatted report and a chart that makes the answer feel obvious. The person who produced it used AI to do most of the heavy lifting, and they are pleased with the result, so is everyone else who reads it. Then someone asks the natural follow-up question - how did we arrive at this number? And they're met with silence - not because the analysis is wrong, but because there is no way to tell whether it is right.
This is a pattern we are seeing more often as AI tools become part of everyday work for people without technical backgrounds. The output looks polished, but the process behind it does not exist in any form that can be inspected. The gap between those two things is where trust quietly breaks down. And as I've argued numerous times before: data insights are useless, even dangerous, if they can't be trusted.
Why the output looks right even when it isn't
Modern AI tools are very good at producing plausible, well-presented output. That is true whether the underlying logic is sound or not. A poorly reasoned analysis and a carefully reasoned one can look almost identical on the page. The chart will be just as impressive, the commentary will be just as confident, the numbers will be just as specific.
If supporting documentation has been written, it's probably not as helpful as you would assume as it will typically describe what the analysis was intended to do. It rarely describes what it actually does, because the person writing the documentation often cannot tell the difference. The intent is in their head, but the implementation is in the AI's output - and the two are assumed to match.
A good way to think about this is remembering the "human in the loop" concept that has become central to the rise of AI - keeping a real person involved in the AI workflow to ensure accuracy, safety, accountability or ethical decision-making. But, this only works if the human can meaningfully evaluate what the AI did.
In a recent piece of work we reviewed, a domain expert had used AI to produce statistical analysis of a fairly sensitive dataset. The code looked plausible, and the code comments were thorough. But buried inside were arbitrary decisions about how the statistical analysis would work - values that the AI had chosen with no rationale attached - and the person who had commissioned the work had no way of knowing they were there, or even that they should be looking for them.
A pattern that causes this: isolated sessions, incremental patches
The root cause is usually structural rather than careless. It comes from how AI chat tools are typically used - one session at a time, each conversation picking up the last one's output and nudging it forward.
The analysis gets built across many isolated AI sessions in an incremental, exploratory way. It works, in the sense that something usable comes out the other end. But it has been patched together across many sessions without keeping a full contextual history, and without ever being focused on creating a working, end-to-end, repeatable process.
That is what causes the whole process to fail. You end up with something that produces an answer but cannot be explained. There is no single artefact you can hand to someone else and say "this is how we got here." Each change lives inside a different chat transcript, each assumption is set in a session that has long since scrolled out of context. Nobody, including the person who built it, can reconstruct the whole thing.
The most important shift: from answers to working out
The single most useful change is in a mindset shift about what you're asking the AI to do. Instead of asking AI for a conclusion, ask it to write the code that produces the conclusion.
This represents a step-change from how most non-technical users approach AI tools. Asking AI for an answer gives you something you cannot verify. Asking AI to write code that calculates the answer gives you something repeatable, inspectable, and shareable. The reasoning moves from inside the AI's head to a file you can open. You can run it again next week with new data, you can add checks and balances, and someone else can independently review it.
The point worth emphasising is that this works even if you cannot read code yourself. If the analysis exists as code, you have the option of asking someone who can. Or, if all else fails, you could use a different AI (model) to act as an independent reviewer, or explain what the code is doing. If all you have is a chat transcript and a final figure, that option does not exist.
Most of us will have encountered this principle long before we heard of AI. In any school level maths exam, arriving at the right answer was not enough on its own. Your teacher wanted to see your working out - not because they doubted you could produce a number, but because the working out was the only way to know whether you understood the method, whether you could apply it again, and whether the answer was the result of genuine reasoning rather than a lucky guess. Marks were awarded for the process, not just the conclusion. The same logic applies here.
A framework for trustworthy AI-assisted analysis
Here are five practical principles, none of which require a technical background to apply.
1. Work toward a process, not a pile of outputs. Every session should be building something durable, not generating a temporary result. The goal is an end-to-end process, set of steps, or pipeline you can run again, not a folder of one-off answers that happen to agree with each other. This might mean instructing the AI to write a plan first, which it can refer back to and update as things progress, or being clear about your expectations on what type of outputs you need.
2. Ask for code, not conclusions. Put the logic somewhere visible and re-runnable. If the AI writes code, the reasoning is inspectable, even if you need help to inspect it. If the AI just tells you the answer, it isn't. Most major AI tools have the ability to write and even execute code, even if you can't.
3. Break it into steps you can check. Ask the AI to show intermediate outputs at each stage - row counts, averages, ranges, the shape of the data after each transformation. And generate these intermediary outputs in formats that you can verify - e.g. .csv files that you can open in Excel. Numbers you can sense-check are far more trustworthy than a single final number presented as the result.
4. Use persistent projects, not isolated sessions. Most AI tools now offer projects or persistent contexts/memory. They keep the AI's understanding of your analysis consistent across sessions and avoid the patched-and-incremental failure mode where every conversation starts from scratch. Add this to an overarching plan, and you've got a workflow that can pick back up where you left off and regain and apply any necessary context.
5. Track what changes. Even basic file versioning helps, like version history in a synced OneDrive folder (but Git is much better). The goal is being able to answer "what changed between this run/feature and the last?" If you cannot answer that question, you cannot explain a change in your results, and you cannot defend the analysis to anyone who asks about it.
What to ask if you're the decision-maker, not the analyst
You do not need to understand the code to ask the right questions. Three questions will quickly expose whether an AI-assisted analysis is verifiable or not:
- Can this be run again with new data and produce the same result?
- Can you walk me through the intermediate steps?
- What changed between this version and the previous one?
If nobody on the team can answer those questions clearly, the analysis isn't ready to rely on. That is true regardless of how good the chart looks or how confident the summary sounds. The biggest hurdle is likely to be the discomfort of asking - it can feel like you are challenging the analyst's competence. But, you are checking that the work is in a state where it can be trusted, which is a different thing entirely.
Summary
So yes, the technology is genuinely capable - AI tools can produce analysis in a fraction of the time that a skilled team can. But capability without traceability is a specific risk. To put it another way, as I said at SQL Bits 2024 in my talk about testing data solutions: If it matters if it's wrong, then you need to be able to prove that it's right.
The risk is biggest for people who do not have a technical background but, importantly, following the advice in this post does not require you to become a software developer. It does require treating AI as a collaborator inside a structured process, rather than a magic oracle you ask once and then publish. And yes, this approach will take slightly more discipline up front, but it will give you something you can actually defend when the questions start.
