Copilots for Customer Service and New Developer Tools - Microsoft Ignite 2023

Microsoft Ignite 2023 just wrapped up, and the first thing I'll say is that the organizing team clearly listened to the feedback from last year, because the event as a whole was significantly improved over 2022.

Value of In-Person Events

First, let's talk about the logistics. Similar to last year, this year's event featured a hybrid format, combining both in-person and digital content.

Microsoft conferences have always been hybrid to some degree, so the question was always whether it was worth showing up in person. For me, the in-person experience is definitely worth it for a few reasons.

Breakout sessions may be streamed, but time for Q&A is also limited. This year, Microsoft balanced this by having all questions (including in-room ones) go through the website. What you can't do online though is immediately ask the presenter a follow-up after the session, or later on in the hub area, which is often the most valuable part of the experience. Not only can this launch a deeper conversation into exactly what I'm trying to understand about the topic, but it often results in valuable connections with people for follow-ups through the rest of the year.

I tried to prioritize in-person features, which included a hub area that was more of a traditional expo hall, albeit a small one with a more subdued sponsor presence. 

The Microsoft subject-matter experts (SMEs) were back this year as well, which meant I was able to make some new connections, catch up with some old ones, and ask a bunch of questions about what's coming to Teams, DevOps, Microsoft 365, SQL, C#, .net, and Azure.

A Focus on Copilot

As for the content this year, what you take away from Ignite will be directly proportional to how interested you are in generative AI (GAI) and Copilot (well, copilots) more specifically. The opening keynote set the tone - Microsoft is now "the Copilot company." 

The reality is a little more nuanced, since there are many different apps that are called "Copilot", but the real intent seems to be that Microsoft, a company that has been building AI services for years, is all in on infusing AI into everything they make and being the platform that powers custom AI solutions for the world.

The Copilot branding is also very deliberate, and I do like the name, because it highlights that this is an assistive technology, not a replacement one. This singularity of vision for the conference's content did help with depth, but breadth definitely suffered because of it. But, given that Copilot was the star of the show, and the main thing that I wanted to learn about here, there was no shortage of opportunities to find out more.

Going into this event, the Copilot that most people associated with the name was the Microsoft 365 Copilot. This is the one with the flashy demos of being able to write a word document, draft an email, or chat using every piece of data in your tenant. There were some statistics thrown around about how much time people feel they saved with it, and how 70% of the early adopters wouldn't give it up.

But the vast majority of the attendees, myself included, haven't had that experience, which has kept me skeptical about how capable the application will actually be. I'm ready to be surprised though, having used the GitHub Copilot for a month or so - when it's right, it's very useful, so hopefully the Microsoft 365 one will be too.

Copilot for Internal Use

For internal use, I'm excited to see how well Copilot will be able to comb through our design specs, emails, Teams chats, release notes, and DevOps work items to help answer questions that I can never seem to remember the answer for, such as "which ice release did we add feature X in?"

The Future of ice Contact Center

During the conference, I learned how the Microsoft 365 Copilot system works, and how we might be able to extend it to bring added value to ice users. Copilot plugins are Teams message extensions, and Graph connectors have been around for a couple of years now, so by leveraging these, I've already got some ideas for new ice features to try out when I get back to the office. 

Beyond the "standard" Microsoft 365 Copilot, there are many other specialized Copilots for specific tasks, like security, the Azure portal, sales, and perhaps most interestingly, customer service. This last one actually matched a lot of what I've been researching for an AI assistant in ice, but built out at Microsoft-scale, and with the full Copilot extensibility plane.

Even though this seems to be driven by the Dynamics team, the intent is that this can be embedded in any contact center application, so this is another feature that I'm excited to be on the early access list for. Again, the Copilot fundamentals apply, but additive features for service, like built-in plugins for ticketing and CRM, make it more targeted. 

I'm already looking at how to extend the solutions it can suggest by including access to things like prior ice contact transcripts; how to add hooks into things like iceReporting, stats, and WFM; and how to embed the tooling in iceBar and iceManager. 

Developer Tools

If you peel back the Copilot layer, the underlying AI platform is also fascinating, and it's actually letting customers - and independent software vendors (ISVs) - design their own Copilots using the same technology. 

Copilot Studio lets you define the interaction flow for a chat application, and prompt flow lets you build and test complex prompts for GAI applications. Both of these look like very useful and powerful tools, made even better by the fact that the low-code web apps sit on top of a pro-code layer that you can open in Visual Studio, version in Git, and import/export at will.

The prompt flow tool in particular had some great debugging capabilities. It fixed the problem I had with the playground "chat with your data" app in the current AI portal, where I wanted to see what got sent to OpenAI but couldn't because it was a black box.

With prompt flow, you can see the content for all steps, including retrieval, generation, refinement, and even re-retrieval/prompting before passing this back to a user. Better still, it looks like these can be integrated like model deployments in AzureOpenAI, so more sophisticated prompt flows can be added without changing underlying code.

All of this new tooling appears to be built on top of the Power Platform, so we'll see how well that works for ISVs and syndication. But it seems like an excellent start, and so far nothing is "hidden," which means we can always fall back to raw OpenAI APIs if we need/want to. I've got a bunch of new ideas I want to try for some near-term ice features, so I'm excited to see how they work in practice. 

What’s Next?

So, the trillion-dollar question now, is whether all of this Copilot hype is actually going to translate into real-world value.

20 years ago, Clippy was trying to help us write letters in Word. Seven years ago, Cortana was going to change the way we used Windows, and Google Assistant and Alexa were going to give us ambient computing assistants that behave like the computer on Star Trek.

All of these came with promises, and users, to various degrees, have rejected all of them because they just didn't do what the hype promised.

This latest wave of AI technology has significantly more power behind it, but it also has a learning curve that users may not be prepared for. Asking the right question of an AI assistant is a little like writing a good Google search, which can take practice. 

The fact that generative AI can also respond with convincingly wrong information is also dangerous. Users will also have to learn to check citations and learn to question things that seem off.

I will say that the capabilities I've seen from this latest wave of AI are beyond anything that has come before them by a wide margin, and large language models really do seem to be able to interpret human intent well, so perhaps this is the wave where these innovations will stick.

As with many things, we won't know until we try, so I expect that by next year's Ignite, there will be a lot of new questions, answers, and maybe even product categories that we haven't thought of yet. Exciting times to be in this space for sure!