Insight Discovery (part 6) – How to define business requirements for a successful cloud data & analytics project
TLDR; The traditional, bottom-up, data modelling approach to data warehousing leads to compromised data platforms that are hard to evolve, expensive to run, and don't meet the needs of the business. Endjin's Insight Discovery process helps you to ask the right questions of the business, so that you can design a data platform that fully meets their needs.
Insight Discovery
This series of posts, and the process that they describe, will help you ensure that your data projects are successful. That the time, money and energy that you or your organisations are investing in strategic data initiatives are well spent and that they deliver real business value.
They describe a different way of thinking, a shift in mindset, in how to approach data projects, that puts the consumer and the outputs front and centre of the process.
In this final post in the series, I'll provide a recap of everything that I've explained, and show you what all of this looks like in practice. And to be honest, there's no complex methodology, or technique to follow. We're just talking about asking questions. But, asking different questions to what you might normally be doing.
Keep asking questions
At endjin, we typically use structured workshops to capture the actionable insights - and there's two ways you could do this. The first way is to cast your net wide and include representatives from different parts of the business in the same workshop. The advantage with this is that you'll get contrasting, sometimes conflicting points of view about what's needed. And that's a good thing as it promotes discussion and prioritisation. However, the downside is that it's harder to narrow in and go deeper into a specific area, if it's not relevant to the majority of people in the workshop.
So the alternative is to have workshops targeted at specific business units, roles, or whatever makes sense for your organisation. You get a narrower and possibly biased set of opinions, but can dig deeper into the specifics of the insights needed.
Neither is right or wrong, and you should find out what works best for you.
I've outlined a fairly simple process so far to discovering the insights we need to deliver, and we take that into our workshops to structure the conversation.
We typically start by setting the business goals. These might be pre-defined, if we're bringing in over-arching, or strategic objectives. Or we might need to understand what they are for that specific group of people.
We then want to capture all the people who can help influence those goals - this might be based around personas, or roles within the team (like the call centre manager), or they might be specific individuals. They might also be people external to your organisation - external stakeholders, customers, partners etc.
For each person or group of people, we then want to capture all the actions they can perform. What can they actually do that will help contribute towards the goal - in the case of the call centre manager, this was managing the number of staff on a shift. This part can sometimes be hard, as you're trying to be very specific about what people can actually do in their jobs. Distilling a job role down into a series of actions and decisions can feel a bit impersonal, but remember - the purpose of this is to help people to be more effective.
At this point, you should have a list of people and actions. It might be quite large. So this is a good time to consolidate and prioritise.
We tend to focus in on what we collectively think are the most influential actions that we've captured to take into the next stage - we want to dig a bit deeper, and want to ensure that we're spending our time on the most valuable things first.
For each of those actions, we now want to capture the questions that we might want to ask. Like "How busy are we now?". And for each of those questions, what evidence could we use to answer them? For example "number of issues resolved per hour". These are our insights - they're actionable, and they're tied back to a business goal.
We've probably ended up with another long list. So it's time to consolidate and prioritise again - this time we can use the axes I mentioned previously "the impact to the business, and the complexity to deliver". Starting with the highest impact, lowest effort insights, we can form our backlog for our data platform.
More than likely, you'll need to repeat this process again and again. This is not a once-and-done activity. You may need to circle back to go deeper into specific areas. You may need to combine outputs of multiple workshops and consolidate and prioritise at a higher level. There's no right or wrong.
You can start delivery as soon as you get your first actionable insight, or you can wait until you've got a backlog that's going to keep you busy for the next 12 months. Either way, you should regularly review your backlog to ensure that the prioritisation is still correct, that the business need is still there.
The hard part isn't the technology
It all sounds easy, right? Well, yes and no. As I stated in the at the beginning of the series - the concept is simple - we're just flipping everything on it's head and working top-down, instead of bottom-up. But putting it into practice can be hard. For a number of reasons:
The business objectives that you need might not be clear. Or they might not be communicated transparently across the organisation.
You're asking people what they actually do in their jobs. They might not be able to, or want to tell you.
Your natural inclination is already to start with the data. And that's probably more so after you consider 1 & 2.
But remember, if you follow this process, you're guaranteed to start delivering something of value. Right from the start.
And whilst the technology absolutely supports the process and makes it easier to deliver insights, as you've probably realised by now, none of this is really about the technology. It's all about people.
Summary
This series of posts has described how to approach your data projects so that they are successful. At endjin, we help businesses go through this process a lot, and whilst it might be hard at first, we've seen how effective it can be, time and time again.
Using the approach that I've described, we've built modern data platforms from the ground up, delivering productionised insights that have been rolled out across an organisation in just a few weeks, so we know it works.
