Launchpad to Success: Building and Leading Your Data Team

TLDR;

High performance data teams transform their organisation by making sure the right people see the right data at the right time to make the right decisions.

• The recipe for an effective in-house data team begins with strong C-suite sponsorship.
• You align organizational goals, adopt a product mindset and position your team as an innovation engine.
• You harness modern cloud native data platforms such as Microsoft Fabric to deliver value rapidly and safely, establishing DataOps from day one.
• The team is a "centre of enablement", democratising data by leading a data skills transformation across the organisation.
• A marketing and communication plan evangelises the transformational impact of your work.
• External help brings in knowledge and develops skills you don't have internally.

Your team is not just a cost center but an integral part of the organization's strategy, driving long-term value.

Introduction

As organizations turn their attention towards AI, leaders are building in-house data teams to take ownership of their data strategy. This guide captures the essentials you need to supercharge your team, modernise your data value chain, and transform your organisation.

This blog is founded on the principle that Data is a socio-technical endeavour. In other words you have to master the cultural, organisational and human aspects as well as the technology in order to be successful in extracting value from data.

C-suite sponsor

Data on its own serves no purpose. To be successful it needs to be woven into the wider business strategy. Therefore, it is crucial to establish a C-suite sponsor who aligns your team's goals with the organization’s strategy. They will lead the transformation, set expectations, and challenge behaviours at all levels across the organisation. This may involve changing incentives and reward processes which don't align with those goals.

Even if you are responsible for data at that level, find somebody else in the C-suite who is willing to work with you, acting as a champion, and leading by example when it comes to the organisation becoming "data driven".

Although developing data and analytics is a strategic endeavour, it is generally delivered by small incremental changes. These small increments add up over time to have transformational impact. It also highlights the new normal for many organisations where "the only constant is change".

It may feel daunting, but it is useful to think about gradually spinning up a flywheel from rest. You transform one person or team, turn them into an advocate, create a case study, and market that success across the organisation. Then move onto the next person or team. Repeat until the flywheel is spinning at speed, and momentum has built to the point where you are getting demand that outstrips your team's ability to deliver (and you have a new problem to solve!).

Secure a regular slot (at least quarterly) with the C-suite to discuss the work your team is doing. This would include topics such as:

• how data and analytics is being integrated into the wider business strategy.
• a living assessment of the overall "data maturity" of the organisation and how the work you are doing is impacting this in a positive way.
• your road map and how it aligns to the wider goals of the organisation.
• an overview the "data products" that your team have put into use and the value that this is delivering to the organisation. This is also an opportunity to ask them to make decisions:
• To make tough decisions about relative priorities in your "strategic backlog" (get validation that you are focusing your limited budget and resources on the highest value data products).
• To allocate more budget and resources to your team.
• To get their commitment to unblock impediments or to sponsor the wider change within the organisation that is often necessary to drive adoption of the data products you are creating.

This line to the C-suite is critical in any organisation from start-up to global enterprise.

A key indicator of success is when senior leaders echo the importance of data in strategy, promote your work, and adopt your terminology and key ideas.

Be goal driven

Align your team’s efforts with the organization’s goals from the outset. Your C-suite sponsor can help you to do this.

You can then build a view of value you are being asked to deliver and "right size" your budget accordingly.

Let's imagine your team is asked to focus on the strategic goal of reducing customer churn by 10%.

After setting this goal, you can estimate its financial impact with assistance from the finance department. For example, reducing customer churn by 10% might improve your organization's bottom line by £2.5M per year. Understanding this value, you can use the "10X rule" as a guideline for the budget allocation. In this case, investing £250K per year to develop, operate and maintain data products that achieve this goal and its financial benefits might be reasonable.

But be careful: make sure you look at the budget as a whole - it's not just the development cost. To be successful, wider changes will be required across the organisation to maximise the benefit. For example, to re-engineer elements of the operating model or to develop new data literacy skills.

The wider leadership team needs to recognise that they are likely to be impacted in this way. They need to commit to working with in partnership with you coordinate organisational change in conjunction with the new data products you release. Factor in a "change budget" component to the overall holistic budget based on the level of change you anticipate.

Taking this approach allows you to clearly articulate the purpose of your team and the value it will generate for the organisation in business terms. It will also set clear expectations around the level of budget that will be required to achieve this.

You may be tempted to tackle many goals to justify a larger team and budget. We would recommend focusing on a small set of goals initially to build credibility and confidence before broadening your scope and budget.

Fundamentally, by taking this goal-driven approach, you will be speaking the right language and reinforcing that data is not a bolt on for the business but something that is integral to its strategy. If you tie everything your team does to the organisation's goals, you will be seen less as a cost centre and more as an enabler for the organisation.

Data teams can often become overloaded with demands. Aligning your team to a small set of organisational goals will allow you to focus limited budget and resources on what matters and shield the team from unnecessary distractions. It also gives your team clarity of purpose.

Act as an innovation engine

Having established the goals, you can engage with the organization to understand their needs and to begin to identify the "data products" they need.

The concept of "data as a product" is adopted throughout the blog. It a core principle of the Data Mesh pattern which is Zhamak Dehghani's thought leadership about how to "deliver data-driven value at scale".

"Data products" are discoverable units of data driven value that are well documented, secure and can be trusted by those who use them. They are accessible and inter-operable, enabling them to be re-used across the organisation for different purposes to generate value. They can be consumed either in isolation or to used as the foundations for new higher order, higher value data products. Hence the concept of a data mesh.

There are many different types of data product. They are often characterised by the "thing" which is consumed such as a report, data API, semantic model, knowledge graph, a data extract to support 3rd party integration or even a simple notification or alert. However it is important to think of the data product as not just the "thing" that is consumed, but as encapsulating everything that allows that "thing" to be delivered, managed and sustained over the long term.

Most fundamentally, we think of the concept of "data as a product" as "user centred design for data". A successful data product is defined by its ability to meet a specific need or solve a problem for a specific group of users. This definition emphasizes the importance of understanding the target audience and their needs.

Therefore, to frame the work that your team will do, we recommend that you perform a "Discovery" process to identify the value streams where data products have the greatest potential to enable your team to achieve its objectives. This enables you to build situational awareness in order to maximise your chances of success.

Use this as an opportunity to begin building deeper partnerships with key stakeholders across the organisation, and determine how they will put your data products into use.

Approach these conversations with curiosity. Assume nothing, and ask lots of questions. (There is no such thing as a "stupid question"!)

If you adopt this open approach, you will be amazed at what you uncover, and the ideas for data products that start to emerge.

We use a structured "Insight Discovery" process to identify actionable insights, this uses a Mural canvas to facilitate goal orientated, collaborative workshops that identify, consolidate, sort, group, and prioritize ideas for data products into a strategic backlog.

This approach helps to position your team as an innovation engine (rather than an order taker). You are taking the initiative to engage with the wider organisation and to identify candidate ideas for your team to work on. These ideas will form a strategic backlog of data products that can be prioritised and then tackled incrementally.

Publish this backlog so that there is transparency of the work you are seeking to tackle and the priority you have assigned. When necessary, gather the relevant stakeholders to re-prioritise this backlog. Never make prioritisation decisions unilaterally. Always involve all stakeholders to reach consensus.

Another objective of this "discovery"process is to identify the stakeholders who are willing to engage deeply and commit resources. Focus on those who share your goals and are willing to partner closely. Remember you are trying to "kickstart the flywheel". We recommend that you prioritise your first few "customers" based on their willingness to engage and partner with you - in product marketing terms, this is equivalent to establishing your "beachhead" and "lighthouse customers". This will not only maximise your chances of success but will also provide a platform from which you can jointly evangelise and raise awareness about the value your team is creating.

Adopt a product mindset

When working through your prioritised strategic backlog, adopt a product mindset.

The reality is that not every idea will make it into production. So, it is a positive outcome if ideas are refined (pivoted) or stopped altogether (fail fast) if they aren’t feasible or valuable. Failure is actually consuming time, budget / energy, not delivering or not learning (and then not sharing that learning across the organisation).

Use this framework to make it clear to stakeholders that not every idea will be pursued, emphasizing the need to leverage the limited resources and budget available to make a high value impact.

The product lifecycle also promotes deep collaboration. You can approach each data product as a multi-disciplinary initiative that integrates the skills of your team with that of the domain experts and end users from the organisation. You should use appropriate product management methods. For instance, you can appoint a "product owner", and adopt a "ship early" methodology to establish feedback loops with end users.

Deliver value early

Get some early "quick wins" in place to demonstrate your team's ability to deliver and to build overall confidence. To achieve this, keep things simple. Avoid betting the future of the team on one large, complex, high risk initiative. Also avoid getting stuck in a cycle of generating "proof of concepts" that are never put into use by the organisation.

Remember, if you don't have real end users, your product is not delivering value. Use the "Discovery" process and product mindset described above to be goal and user aligned.

Use a prioritisation process to identify the quick wins. We like to use a quadrant chart to help clients prioritise candidates for data products - the sweet spot being the high business impact / low technical risk data products first:

Remember that you are trying to take the wider organisation on a journey. By starting with simple data products you can help them to build their data literacy and understanding. With each data product providing the building blocks for the next. For example:

• A - the first data product could be an interactive Power BI report that provides descriptive analytics.
• B - having built confidence, a further data product builds on this data set by adding more complex DAX measures and use the "Key influencers" visual to layer in diagnostic analytics.
• C - build on the historic information and diagnostic insights to train a machine learning model to deliver predictive analytics.
• D - develop a prescriptive model that analyses the data in order to make a recommendation or to take an automated action to achieve a specific goal.

The layering of data products in this way is consistent with Data Mesh principles.

By taking small steps in this way, you are creating opportunities to create regular case studies. Put the effort in to capturing these case studies. Be sure to interview the end users of the data product when it is fresh, getting them to talk about the impact it is having on the way they work.

Focus on DataOps

With the discovery process having identified the functional requirements for a data product, your team now have the task of designing the architecture to deliver it.

To frame this design, articulate the non-functional requirements specific to that data product - in other words think about concerns such as:

• Total Cost of Ownership (TCO)
• Quality
• Observability
• Maintainability
• Flexibility
• Scalability
• Security
• Accessibility
• Repeatability

Remember to apply the "goal" orientated approach to this process. This is about choosing the appropriate tools and methods to achieve a particular goal. It is NOT about imagining all possible future goals in order to deliver the "one true architecture". You should be mindful of the wider considerations, taking advantage of (and evolving) previous decisions, but the first rule is to make the right choices for the clear-and-present goal.

This approach designed to enables the rapid and safe delivery of value:

• By rapid we mean taking humans out of the loop through automation and adopting tools that boost productivity.
• By safe, we mean removing risk and reducing costs.

In other words the team should be seeking to build, test and release new features in a repeatable, reliable, autonomous manner with a high degree of confidence and with little or no ceremony. In software engineering the term for the capabilities is DevOps. In data projects, the application of DevOps practices is known as DataOps.

This DataOps infinite loop is illustrated below with specific activities that are performed at each stage in the loop called out:

Equipped with a clear understanding of non-functional requirements you now have the framework with which to make an informed choice about the specific DataOps capabilities (platforms, tools, processes and skills) required to succeed. You are likely to have many of the the capabilities in place already, but you may identify gaps that need to be addressed. This may mean choosing to activate un-used features in existing platforms, introducing new processes or integrating new tools into your infrastructure.

We recommend documenting these decisions using Architectural Decision Records (ADRs) as a means of allowing other stakeholders such as colleagues from the IT or security team to engage in the process. Over time you will be able to use these ADRs in a "cookie-cutter" fashion as common architectural patterns emerge that can be re-used across multiple data products, or as we like to say "make future you thank past you".

Focus on SaaS and PaaS solutions whenever possible and minimise the number of moving parts you need to integrate. While this might seem like a compromise, it prevents the headache of integrating and managing disparate components from a diverse set of vendors, which can result from choosing "best of breed" technologies and open source frameworks that require significant work at the infrastructure level to deploy them.

SaaS products such as Microsoft Fabric are attractive because they offer a huge breadth of functionality to end users "out-of-the-box". You can just buy the service, with no complex deployment and configuration requirements.

However, there is still likely to be supporting infrastructure you will need to address cross-cutting concerns. This might include source control, logging, observability, secrets management, and 3rd party integrations.

Consider using an external technology provider who do dozens of deployments a year to help de-risk the process of scoping, designing and establishing new infrastructure capabilities. They bring the expertise and IP that will enable your team to become skilled up and productive in a short period of time.

Recognise that you won't necessarily be able to fulfil all of your DataOps ambition on day one. Get the fundamentals in place, then build up the remaining capability incrementally as and when required. Use a heat map to be transparent about your maturity and to show stakeholders how you have addressed this over time.

Establish a positive culture

Another aspect of establishing any new team is to determine the type of culture you want to promote.

While culture tends to be inherited from the wider organisation, you can still influence your own team. Key cultural attributes we see with high performance data teams are:

• A safe environment - this is fundamental if the team are seeking to adopt a product mindset. The ability to be open and honest about what isn't working, to see "failure" as an opportunity to learn.
• • Looking outward - they understand that they are part of a wider organisation, the role they play in helping that organisation to achieve its goals and the flow of value that they can facilitate. Their goal oriented approach keeps them in tune with the shifting needs of the wider organisation. They are able to adapt (and lead others in adapting) to meet those shifting needs.
• Lifelong education - personal development is valued and space is actively created for training, mentoring, collaborative problem solving and career advancement.
• Self awareness - there is no such thing as a perfect team, but good teams are "self aware". They are honest about their weaknesses and where they could improve.
• Continuous improvement - teams are always a work in progress. They are self organising with a focus on constant improvement. They set aside time to incrementally work towards achieving perfection. They recognise that perfection is a moving target that they will never achieve thanks to the constantly changing environment in which they operate. A focus on DataOps is a key way of driving this approach.
• Productive partnerships - teams don't live in isolation. They need to interact with other teams in order to succeed. They recognise the value in forming productive working partnerships with other teams both internally and externally.

We would recommend working with the team early in its lifecycle to explore these culture shaping attributes and others that you want to consider. By getting the team involved in shaping the culture, you promote stronger engagement.

Develop a plan for skills

Develop a skills map and career pathway for your team. At endjin we have a career pathways framework that takes an evidence-driven approach to guide personal development across a broad spectrum of skills. We seek to develop well rounded individuals who master not only the technology, but also grow capabilities in other areas such as commercial acumen, leadership, team work and client relationships. We recommend taking a similar approach - data is a socio-technical endeavour, so those "softer skills" are as important as the technical ones.

Extend your skills development programme to the wider business. Successful data teams "power up" the whole organisation by helping them to acquire new skills, to gain data literacy and to build confidence to build and own data products. The level of opportunity (and the business transformation required) should be viewed as being equivalent to the historic adoption of email, spreadsheets or the web browser as ubiquitous transformational technologies.

With modern SaaS platforms such as Microsoft Fabric, and specifically the "low code / no code" tools that Fabric provides such as Data Flows and Power BI, the barriers to adoption are lowering. Your team should be viewed as a "centre for enablement" that is powering up the whole organisation, rather than as a "centre of excellence" which will quickly become a bottleneck if it is the only place in the organisation where data products can be delivered. Look to Data Mesh for some inspiration about the direction of travel of your organisation and to formulate a vision where new data products are built and owned by the domains, with your team acting as an "enabling team" who are called on when some heavy lifting or deep technical skills are required.

Communicate, communicate, communicate

As team leader, you should be acting as "chief data evangelist" communicating to the c-suite, to your peers, to the whole organisation and with your team on a regular basis.

The need to communicate is essential to securing the success of your team and ensuring their efforts are appreciated. You need to constantly reinforce:

• The purpose of your team.
• The goals you are targeting.
• What you are focusing on now.
• What you have delivered recently and the impact that it has had on the organisation.

Opportunities to communicate are baked into the content above: at quarterly sessions with c-suite, during "discovery sessions", as part of sharing the strategic backlog, whilst engaging with end users throughout the product lifecycle, as you collaborate with technology peers through ADRs and DataOps initiatives and through the training sessions you may run to up-skill the whole organisation. Each is a key part of building momentum in the "flywheel".

You should augment this with more formal communications such as publishing case studies. If your case studies can be shared externally, you will often get help to do this from product vendors, who want to amplify your success and gain mutual benefit by basking in your reflected glory, to showcase how their products help you achieve your goals.

Another activity that can really help to support your communications strategy is regular blogging. As with case studies, where the content is suited to sharing outside your organisation, you can make your blogs public through channels such as LinkedIn. This can often help you break out of organisational silos and help spread your message to senior leadership.

We find that a lot of data teams have a great story to tell, yet struggle to set the time aside for internal marketing. Whilst this may not come naturally, regular communications are critical to sustaining long term commitment to the work you are doing and helping the wider organisation to maximise the data driven opportunities that are available to it.

Conclusion

As data leaders, you are the architects of change, the catalysts for innovation, and the champions of a data-driven culture.

Your mission is to ensure the right data reaches the right people at the right time, empowering them to make informed decisions that drive the organization forward. This task may seem daunting, but remember, every small increment adds up to a significant transformation.

Building an effective in-house data team requires strong C-suite sponsorship, alignment with organizational goals, a product mindset, and a role as an innovation engine.

By fostering a collaborative and positive culture, focusing on early wins, and embracing simplicity through SaaS and PaaS solutions, your data team can drive significant value and transformation.

Communicate your successes, however small, and let them serve as a beacon of what’s possible.

Make data an integral part of your organisation's strategy, not just a cost center, and watch as you reap the rewards.