Insight Discovery (part 2) – successful data projects start by forgetting about the data

TLDR; The best way to start a data platform project is to work from the business goals down, not from the data up. Start by understanding what the business wants to achieve, and what decisions and actions need to be taken in order to achieve those goals. Then work out what insights are needed in order to make those decisions and take those actions. And only then think about what data you might need to deliver those insights.

Insight Discovery

This series of posts, and the process that they describe, will help you ensure that your data projects are successful. That the time, money and energy that you or your organisations are investing in strategic data initiatives are well spent and that they deliver real business value.

They describe a different way of thinking, a shift in mindset, in how to approach data projects, that puts the consumer and the outputs front and centre of the process.

In the first post in the series I explained why we need a different way of thinking in how we approach data projects, to avoid building brittle, costly and compromised solutions.

And in this post, I'm going to introduce endjin's Insight Discovery process, starting with the single biggest mindset change that will help you deliver successful data insights:

A successful data project starts with forgetting about the data.

Understand the business objective

That probably sounds entirely counter intuitive, but if we go back to the story I described in the first post, what's clear is that by starting with the data, working bottom-up, we ended up with a solution that promised self-service and flexibility, but actually delivered a compromise that didn't serve the business need and therefore didn't add the value or impact that it should.

So, if working bottom–up from the data doesn't work, then the obvious thing to do is to work top-down. Not with the data, but with the business objectives - what goals are you trying to achieve, and what are the decisions and actions that can be taken towards achieving them?

If we explore that story in a bit more detail, our business had identified that they could and should be using data to make informed decisions. They wanted to become more "data driven".

But driven towards what?

Decisions need to be made in context. What are the strategic objectives that led to this data platform initiative?

Is it that the business is aspiring to grow, or introduce efficiency and cut costs? If it's about growth, is this through organic growth or acquisition? If it's about organic growth is this through new customers, or improving customer satisfaction and reducing churn, so you can up-sell new products and services?

Of course, data can ultimately help you with all of this, it can help you make the right decisions, and take the right actions. But you need the context in order to define what is "right".

So, before you do anything else, before you consider what data you might already have, or think you'll need - make sure you understand the objectives of the business that have led to this data project. Why are they investing in this project, platform, or initiative? What are they hoping to achieve? And what is the planned strategy to achieve it?

You might find at this point that the data you have isn't what you needed at all.

Of course, the business objectives may change at different organisational levels - departments or business units might have their own specific objectives either inline with, or in addition to overarching strategies and goals. And that's absolutely fine. But make sure that at whatever level you're focusing on, everyone understands these goals.

Start with outputs, not data

Once you’re clear on the goals, you can work backwards from there. And yet, we're still not ready to start talking about data.

Because the truth is that people don't really care about the data. They care about what the data is telling them.

They care about the insights, or knowledge, or learning. And they care about those things because they help them in making decisions or taking actions. Especially when those decisions and actions are tied back to a bigger business goal.

So, instead of asking people what data they need, we should be asking what can they do within their role, in order to help meet the business goals that have been agreed. Because fundamentally, anyone's job at any level can be boiled down to a series of decisions to be made, or actions to be taken. So we can find out what those decisions and actions are, and then work backwards from there to understand the insights that are needed. And at that point, we can start to think about what data we might actually need to deliver those insights.

An example

Let's see that with an example. Let's look at a telecommunications provider that has a problem with customer satisfaction. They might have identified that the customer service call centre is a key business unit that can influence customer satisfaction. So the call centre managers are a key stakeholder in your new data warehouse.

How can you ensure that your data platform delivers value to them - that it provides the right insights, that it will actually help them?

Ask them what they can do that will contribute to the goal.

How can a call centre manager improve customer satisfaction? They could try to keep the call queue length as short as possible so that customers don't get fed up waiting. How can they keep the call queue length as short as possible? They could increase the team size for a shift if the phones queues are going to be busier than previously expected.

This is a specific action that can be taken that will have a positive impact on a business goal.

In the original story in the first post, we started by looking at the data. The data we had, or the data we thought we'd need, and built a model around that that would hopefully provide the right information. It was a data-centric, bottom-up approach to building our data platform. And that caused problems.

We've now flipped everything around. We haven't even talked about data yet. We've focused actions and decisions that the business can take to achieve their goals.