Microsoft’s new Copilot will change Office documents forever

Microsoft’s new AI-powered Copilot instantly summed up my social yesterday (the meeting was with Microsoft to discuss Copilot, of course) before listing the questions I asked just seconds ago. I’ve watched Microsoft showcase the future of work for years with concepts around virtual assistants, but Copilot is the closest thing I’ve seen them come true.

John Friedman, vice president of design and research at Microsoft, said in an interview with Microsoft the edge.

I was talking to Friedman on a Teams call when he activated Copilot halfway through our meeting to perform his AI-powered magic. Microsoft has Glamorous marketing video It shows the potential of Copilot, but seeing Friedman demonstrate it in real time across Office apps and in Teams made me convinced it would forever change how we interact with programs, create documents, and ultimately, how we work.

Copilot appears in Office apps as a handy AI chatbot on the sidebar, but it’s much more than that. You could be in the middle of a Word document, and it would pop when you highlight an entire paragraph—just like how there are Word UI prompts that highlight your spelling errors. You can use it to rewrite your paragraphs with 10 suggestions for new text to freely move through and edit, or you can have Copilot create entire documents for you.

This adaptability is what sets it apart from Microsoft which just pushes ChatGPT to a sidebar in Office. Not only does Copilot offer a chatbot interface, but you can use it to command Office applications such as Excel and PowerPoint. If you’re looking at a slide deck and want each title to be orange instead of blue, just ask Copilot instead of having to dig into PowerPoint’s features.

In Excel, you can have Copilot create a PivotTable, create a graph, or just help you make sense of the rows and columns of the data in front of you. “One of the ways we start with Kopilot is to help analyze and understand the data,” Friedman says. “You can ask Copilot what it makes of the data, you can have Copilot graphs based on the trends it sees in the data, and you can insert those trends into a spreadsheet.” Excel also has Copilot’s “show me” feature that will let this AI let you know how it just completed a command so you can improve your knowledge of Office.

Microsoft seems to be slowly building on the vision it had to help Cortana or even Clippy decades ago. “I love that our heritage is filled with products that try to adapt to people,” Friedman says. “Copilot shares some similarities with some of the things we’ve done in the past, but it’s much more capable, it’s humble, and it’s there to serve things that help you save time.”

Microsoft has customized this Copilot system for each Office application, so there are different ways to command it. Friedman showed me how Copilot can help you compose emails in Outlook, offering drafts of short or long messages with options to change the style. It even works in the mobile version of Outlook, which got me thinking about ways this can speed up work on the go.

“Outlook for mobile is the first place we’re making a big push,” Friedman explains. Outlook can summarize all your email messages on the go, create drafts, and generally make it easier to sort your inbox. But imagine creating entire Word documents from your phone without having to type on a tiny on-screen keyboard. “We’ll have more to talk about mobile in the coming months,” Friedman says. But you can imagine where things will go.

As impressive as Copilot is, we’ve seen the myriad ways big language models can fail, including introducing racial or gender bias into text and simply making things up. These traits are troubling enough in a search engine, but when you’re talking about Excel (which arguably powers the global economy) or your email inbox, it’s a whole different level of ethics, privacy, and data concerns.

“It makes things right often but not all the time,” Friedman admits. “In user experience, we do things like don’t submit anything until you’ve read it, or encourage you to try again, edit, and ignore.”

Microsoft also has a number of warnings within Copilot that appear as you use it. In PowerPoint, you’ll see a message that says, “Content has been generated by artificial intelligence and may contain inaccurate information for sensitive material. Be sure to check the information.” Elsewhere, there are claims that say, “AI-generated content may be incorrect.” Microsoft is trying to design the system in a way that reminds you of that You are on responsible for.

“We give you tools to report it when it’s wrong. We create quick suggestions to help you write good claims. Everything we do in the user experience is make it a conversation and give you agency,” Friedman says.

We’ve seen what happens when an error occurs in Microsoft’s Bing search engine. The AI-powered chatbot has hallucinated on multiple occasions, and Microsoft had to set limits to control its outbursts. In one conversation with the edge, Bing even claimed to have spied on Microsoft employees through webcams on their laptops and manipulated them.

“Everything we learn from Bing in the preview helps us mitigate these risks,” Friedman says. “We’re applying all of that learning and thinking to the co-pilot as well.” Microsoft also got its start small by rolling out its Copilot. It will initially be available to just 20 companies before Microsoft opens it up to more when it’s ready. Microsoft also starts with enterprise customers first before bringing it to consumers.

“We feel very good about what we have as a starting point, but we don’t know yet if it performs the way we want it to and really helps empower people to get their jobs done,” Friedman says. “We’re going to iterate quickly, we’ve built quickly, very quickly. But we pause, we learn a lot, and we’re going to update very quickly. Our plan is to move as fast as we can to scale the business in a thoughtful and responsible way and make sure the experience is great.”

Is Microsoft moving too fast, though? Google announced its AI features for Gmail and Docs earlier this week, and the AI ​​race has many experts concerned that the tech giants are not properly considering the impact of these new tools.

“In our minds, we’re measuredly fast,” Friedman says. “We’re thoughtful because we’re rolling it out to 20 clients and working side by side with them.” Despite reports of Microsoft laying off an ethics and community team that taught employees how to make AI tools accountable, Friedman says Microsoft is cultivating people working on these concerns. “In terms of our investments in ethics and AI, we’ve grown more and more ethics and AI experts across all of the product teams working on these things,” Friedman says. “We have to scale a lot more, so we’ve been investing more and it continues to grow year after year.”

Microsoft knows Copilot isn’t perfect and that it’s going to take some time to get there. While it impressed me during a Teams meeting summary, it could easily have confused my voice with someone else’s if I had a bad microphone, or Outlook would pull the wrong summary in an email thread. There are significant challenges ahead, but Microsoft hopes that the work it does to make it easier to edit responses, correct sources, and release feedback will eventually improve the system.

“We know AI gets things wrong, we know it hallucinates and we know it does it with confidence,” Friedman admits. “We’re continuing to work on making it better at doing it less, but also the user experience really empowers people and puts them in the driver’s seat.”

For all its challenges, the future of Copilot won’t be just text-based creation either. Microsoft has a clear vision of using Copilot to create images, video, and more once large language models can handle these features well.

Microsoft has already integrated OpenAI’s DALL-E model into its Designer app, which allows people to create images based on text. The designer will also help PowerPoint select the best images for AI-generated slides. “We’re going to bring Designer further into Copilot so you can change things inside Designer,” Friedman says. “The designer elements you saw today are just scratching the surface. I fully suspect we’re going to use Copilot to do amazing multimedia stuff.”

So where else could we see the co-pilot appearing? I asked Friedman about Windows integration. “We’re looking at all kinds of places and ways to expand [Copilot] Outside. I believe this is the next major wave of computing and it will change the way we work with all devices in the years to come,” says Friedman.

The future will also include a multiplayer experience for Copilot as well. Loop components, one of the biggest changes to Office documents in decades, are now available in Teams and Outlook. Episode components, Microsoft’s trademark Fluid work, are collections of collaborative text or content that can live independently and can be freely copied, pasted, and shared.

Now, imagine copying a text Loop component into an email and having multiple people edit it and interact with the co-pilot. “In the component as we’re editing, the conversation is a clickable history of the content being created that you can go back and forth,” Friedman says. “The cool thing about it is it feels like a whole new mental model with how to work as a co-pilot with a group of people.”

All of these Copilot features for Office and Microsoft 365 feel like they’ll forever change the way we work and communicate, especially as these big language models evolve in the years to come. Microsoft’s push to integrate this AI in depth into its products could have a lasting impact on the job market.

“Every time there is a new technological advance, there are opportunities and things that we have to take into account,” Friedman says. We believe that infusing this AI will create new job opportunities in the long term, and increase job satisfaction in the short term. We expect that it will change the nature of many jobs and create new jobs that did not exist before. That’s why empowering people and building this co-design ecosystem is so important to us.”