Decision Makers Guide to Microsoft Fabric - Microsoft Fabric and Generative AI | endjin

Ed Freeman Ian Griffiths 18th July 2023

Decision Makers Guide

Welcome to the second part in a new series of interviews with real-world Decision Makers (CTOs, CIOs, Heads of / Directors of Software Engineering, Data & Analytics) about how they manage their strategic roadmap and evaluate new technologies to simplify their portfolio, deliver better outcomes for stakeholders, or give them a competitive advantage. In a 3-part interview we talk to Tom Peplow about his assessment of Microsoft Fabric. In this part we talk about the relationship between Microsoft Fabric and AI.

TLDR;

Tom Peplow, Principal & Senior Director Product Strategy at Milliman, chats with Ian Griffiths & Ed Freeman from endjin about how AI could impact the insurance industry. Tom believes that AI is going to impact the future of business, and that it is up to IT leaders to help their organisations leverage the power of AI to better serve their customers. He discusses the relationship between Microsoft Fabric and AI, and how organisations can create a trusted AI environment where they can experiment with their own data to build AI-powered solutions. Tom believes that a good strategic IT department should be thinking about how to federate information across their organisation and with the outside world in order to train a Copilot on top of that information store. He believes that people inside an organisation will be 10x more effective at their jobs if they have access to AI-powered tools, and that there shouldn't be any fear around this. However, Tom is quick to point out that IT leaders need to figure out mechanisms for enabling businesses to grow quicker, leveraging tech, and not look for reasons to slow it down. He believes that it is important to build ethical AI frameworks inside of an organisation, in order to provide guidance and protection around internal intellectual property. Tom acknowledges that there is a skills gap when it comes to AI, but he is optimistic that this will get exponentially easier to harness as the technology matures. IT leaders need to be proactive in helping their organisations leverage its power to better serve their customers.

The talk contains the following chapters:

00:00 Introduction
00:24 Empowering users and AI
01:32 Consuming corporate data with AI
04:05 Data isolation and protection in AI
06:48 Upskilling and AI

Microsoft Fabric End to End Demo Series:

Microsoft Fabric First Impressions:

Decision Maker's Guide to Microsoft Fabric

and find all the rest of our content here.

Transcript

This is the second part of a discussion about what Microsoft Fabric means for existing investments in the Microsoft Azure data and analytics platform. In this second part, Tom started to talk about the relationship between Microsoft Fabric and AI.

Ian Griffiths: Is there some fear that an unintended consequence of putting more power in the hands of end users is that they'll be able to make an expensive mess of things?

Tom Peplow: Yes. Let's play this through. If we think about fear, you've got one level of technologies that makes things simpler. But on the other side, someone's going to fill the void left with that simplicity. What could you be doing to best support the business? If you're focusing on that, your time is now available to do other things. There are two disruptions happening at the same time, and I would imagine that the timing is well thought through; Microsoft's strategy was laid out in Satya's book when he took over, and they've been executing against it with a drumbeat. It's been fantastic to see.

But to be able to leverage the capabilities of AI, you need data. And you can see the value it's generated; Large Language Models (LLM) are generating huge amounts of insight based on the Internet's worth of data, but it's void of anything a company's ever done because it's locked up inside the enterprise. Lots of companies are thinking, how do we connect the dots between what we've got internally and what's on the Internet, so that we can help our users in our business service our customers better? If you've got a really good strategic IT department, they are thinking about how to federate information across their organization and with the outside world, and then train a Copilot on top of that information store.

We can then start to leverage the insights that we've got internally with the data that we've got from our vendors and publicly available. People inside our organization are going to be 10x more effective. And that's what AI is there to do. It's there to make people in the business more effective at their jobs. So, there shouldn't be any fear. I'm optimistic because I'm an optimist. People don't like that about me sometimes, I'm excited because I have all this talent solving these trivial problems. They're now trivial, they weren't trivial, let's face it. Integrating all these things together was a hard problem. We need to spend millions of dollars doing it. But someone spent billions of dollars doing it. So why would you do it again? It's a tough question. And then you go on to what can I do now? All that talent is not doing that. And then you look at what they could be doing that.

It is really interesting where what we could do, but the data foundation has to be laid right for you to be able to leverage all the cool stuff that's coming down the pike. And I think it has been really good to see Microsoft's thoughtfulness around how they've gone to market with the data products in AI products. Because it also helps those users who are given new tools to learn.

If you think about how quickly end users can start to leverage all this new tech when it has got Copilots embedded inside it. The diffusion of innovation from the early adopters through to the laggards in the business is going to be super tight, which means you are going to get a lot quicker value realization. Which means that you can invest more because you must wait less time for you to get an ROI.

So, if you are quick and decisive and you get on with things, your business could flourish in ways that others will not. So, then IT is back to being a key enabler of growth by an organization. And I think that is super exciting. Yes, there's fear, but I think IT leaders like me need to really try and figure out mechanisms for enabling businesses to grow quicker, leveraging tech, and not look for reasons to slow it down.

Ian Griffiths: Are your customers thinking hard about AI in this world? And do they have any concerns about whether you are going to build models on their work, and that's going to leak their expertise? Because that is becoming a concern with people wondering where does the Intellectual Property for things like DALL·E come from? Are there equivalent concerns in your world? And how do you see that playing out?

Tom Peplow: Yes there are, I will give you a simple example; job one is to try and keep your customers data safe, separate from your own data. That is kind of table stakes. That is super important governance stuff. That is now, foundational within Microsoft Fabric. It is important in every person's data product. And, if you look at the agreements you sign when you use Azure OpenAI, it is clear that the models you're training on your data are your models and your data. That is the same for our business too. It is clear that where the separation of Intellectual Property lies. Milliman is a business that has offices all around the world, with customers all around the world. with different agreements. So, the data governance question is what information we can train models on, requires us to be able to understand what contracts we signed with our customers.

A lot of what we do becomes public domain. It's not like insurance policy terms and conditions are secret, so there's a lot of what we can train models on top of which is safe to do and a lot of which we can train models on top of, which we would not be safe to do. If we've helped a company decide to buy another company, it's probably not a good idea that becomes part of the model that we trained. So very tight data governance building that into existing processes because every company is already doing it. We used to work in investment banking, it is very clearly separated what you're able to do and what you're not able to do. Who you're able to buy shares in and who you're not able to buy shares in. So, all that stuff exists, but now it needs to be codified so a computer can understand how to do it so that it can then do the right thing when it's training models.

That's where I think a lot of the focus needs to move to with this is; it's obvious you need to be able to do it, but you need to be able to do it safely. So, building ethical AI Frameworks inside of your organization to be able to safely start to connect the dots with data governance tools building an enterprise-wide ontology of what information you have, buildings semantic models, all those things. And it's going to become important for differentiating yourself in the market soon that you're able to do that well.

Ed Freeman: I presume similarly to Microsoft Fabric being a new data platform that say your Actuaries need to get familiar with. From the AI side, we've got new concepts and principles that your software engineers are going to have to learn, like prompt engineering and how to do that correctly, and get the most value out of it. So, you've got upskilling required in almost all of your personas, right?

Tom Peplow: Yeah, definitely. That's why it's exciting. It's not that your job is going to get more boring. The things that you didn't want to be doing anyway are not going to be done anymore. As Satya says, the drudgery of the work goes away.

And it's not that you're going to kick back. I would love it to be honest, if I could work 10 hours a week and deliver the same value that I was before. But the likelihood is that they're going to want me to do something else for the other hours to deliver more value. So training is super important making sure people realize that there's a way to move into that future.

GitHub Copilot for software engineers is a really useful tool; it makes it very easy for you to start to learn those techniques and you're in charge in a safe way. It's like you're asking it to help you refactor some code You're learning the right ways to ask it questions to do that and then when the business comes to you and says could you build something similar for me? You know what good looks like because you've been using these tools inside of your tool you use every day. That's how that innovation diffuses through the business. If you think like that what we were doing as software engineers, Ian and I, 20 years ago. It's a lot easier now than it was 20 years ago, for sure.

And I think there's a hockey stick curve. It's just going to get exponentially easier as the technology gets exponentially better, the ability to use natural language to ask a computer to do things really levels of playing field for everyone to be able to the value that was locked up in your software engineering department. But it doesn't mean the software engineers are not going to be doing good things, it just means it can be in different things with much higher impact to the business.

Ian Griffiths: In the third and final part of this discussion. We look at how Milliman resolves the tension between the long-term outlook that is required in the insurance industry, and the fast pace of evolution in cloud platforms.