Last quarter, a product team pivoted their entire roadmap based on an AI-generated insight. The result? A disaster. The '32% conversion potential' they cited was based on a biased definition of “users”. Welcome to the Wild West of AI Analytics. It has helped us prioritize speed over accuracy, and the consequences are stacking up.
Note: In this post, I use “Data Scientists” as shorthand for analytics professionals, partly for SEO reasons :)
The Standard of Care
We often reminisce about the pre-AI era of data science, but we should view that time as the “Standard of Care” rather than just the “good ol' days.” The workflow was rigorous by design:
- Gather business requirements.
- Define the scope of what analytics can and cannot solve.
- Extract and model the data.
- Compile the narrative.
- Quality check from both technical and business perspectives, identifying bias.
- Present findings with clearly defined assumptions.
Quality checks and sanity checks weren't optional tasks; they were the anchor of the analyst's role. If an analyst missed these, they were held accountable.
Brave New World of AI Analytics
AI Analytics tools are quickly becoming the first stop for business partners looking for answers. The questions range from simple investigations like, “Why did shopping cart conversion drop last week?” to more prescriptive ones like, “What friction points should we address this quarter?” They are also being used for presentation-ready visualizations, autonomous metric monitoring, and recurring summaries. Large companies are building custom capabilities, while smaller ones often rely on fragmented toolkits.
This has changed the role of Data Scientists as well. As business partners become more self-sufficient, Data Scientists are increasingly called in when the questions become too complex—or when the AI cannot get to a credible answer on its own. Yet amid this rapid shift, one topic receives remarkably little attention: the quality of the insights being produced. As with much of the AI boom, success is often measured by adoption, speed, and usage. Quality control, by comparison, has largely been reduced to a disclaimer at the bottom of the screen: “AI can make mistakes.”
Why Every Company Needs to be Wary
Compare software development, where AI tools have deep integration, to insight development. Code requires rigorous QA, testing, and legal review. Conversely, an AI-generated insight claiming “64% of users showed desired latent behaviors” often goes unchecked. Who verified that 64%? Was the definition of “user” or “latent behaviors” biased? The range of errors can be from simple data errors and definition errors as above to causal errors, scope errors and presenting insights with a level of conviction that the data does not support.
As I mentioned earlier, analysts were held accountable if they missed out on the data quality checks. Can we hold business partners using AI Analytics tools accountable to the same degree, even if analytics is not their core competency or when they can blame the AI tools for not warning them?
Are Companies Being More Responsible With AI Analytics Usage?
Short answer: Yes. Compared to the debacle of token maxing, the response I have seen from companies on how to address AI-generated insights has been surprisingly mature. Without naming them, here are some ways I have come across on how different companies have tried to temper the AI Analytics explosion:
- The Human-in-the-loop Model: Analysts remain the primary gatekeepers, using AI to scale their own efficiency. This remains the gold standard.
- Insight Governance: Allowing access to tools while analytics teams play the role of “governor” before final decisions are made.
- Tiered Rigor: Not all insights require the same scrutiny. Simple descriptive metrics can be self-service, while product-driving strategic insights require manual audit.
- Built-in Controls for AI Tools: Building controllership within AI tools is tougher than it sounds, especially since AI tools struggle to know when to say “no” to business requests. However, this approach is becoming a go-to for several companies who do not want to give up efficiency gains from AI analytics tools.
Eventually, bad insights will lead to bad investments and more companies will start questioning if they need some kind of audit of insights being generated with AI.
Insight Controllership: The New Frontier for Data Scientists
As the initial hype of AI adoption settles, we are seeing a crucial shift. Insight Controllership is emerging as the next evolution of our field.
Do you know where AI makes the most mistakes? Can you build quality checks into automated workflows? These are the questions defining the future of data science.
Insight Controllership isn't a defensive role—it's the competitive advantage of the next decade. The companies that win won't be the ones that use the most AI; they will involve data scientists to design systems in which the appropriate level of validation happens automatically, while humans intervene where judgment is required. Data scientists, you are no longer just analysts—you are the Architects of Trust. The genie is out of the bottle; it's time to take control.