Home › Forums › General Chat › MJF Chat › Why it's not too early to start learning about Microsoft's Mixed Reality apps › Reply To: Why it's not too early to start learning about Microsoft's Mixed Reality apps
Audio playback can be found here.
Mary Jo Foley: 00:05 Hi, you’re listening to Petri’s MJFChat show your Petri.com Community Magnate. That’s me, Mary Jo Foley, is here to ask industry experts about various topics that you, our readers want to know about. So today’s MJFChat is all about why it’s not too early for you to start learning about Microsoft’s mixed reality apps. And my guest today is Lorraine Bardeen, who is the Microsoft General Manager of Dynamics 365 mixed reality apps. Welcome Lorraine. Thank you so much for joining us.
Lorraine Bardeen: 00:42 Thank you so much.
Mary Jo Foley: 00:44 So we’re going to go right in here. I’m going to talk about, what we’ve learned recently about HoloLens 2, and ask you a few things around that, specifically about the apps. A couple of weeks ago, Microsoft announced its second generation HoloLens device. But to me the real stars of that show where the new Azure services and the Dynamics 365 mixed reality apps, some people think that mixed reality is so far away from being usable in the real world reality that they don’t really need to start learning about it yet. But is this really true? I think smart IT Pros need to start figuring this out today and it’s not too early for them to be trying it out and learning about this technology. So that’s what we’re going to discuss. So before we get into that, Lorraine, I wanted to do a quick level set for people who might not be as familiar with mixed reality at Microsoft as I am. I remember when I saw the first prototype of HoloLens 1, I think it was back in 2015, Microsoft was showing off both consumer and business apps for the device. But over the past couple of years, I feel like the focus has really shifted to business and especially in the mixed reality space from Microsoft.
Mary Jo Foley: 01:58 So I want to ask you why that has happened and why did Microsoft move mixed reality apps under dynamics, which is a CRM and ERP brand. I get this question a lot. So dig in.
Lorraine Bardeen: 02:13 Great. Thank you so much Mary Jo. So yeah, I remember that launch day like it was yesterday too. And you know, my team is full of application developers, creative folks, product managers, etc. And we love making things that people will use. And so on that launch day, you’re right, we showed off a lot of different applications to really expand how people would think and to give examples of what people could possibly do with the types of incredible new technology that the HoloLens represented. You saw all sorts of things, communication, collaboration, games, et cetera. And then we’ve spent every minute since that day learning and paying really, really close attention to the feedback we’re getting, certainly from developers and absolutely from companies.
Lorraine Bardeen: 02:56 We’ve had the opportunity to work with hundreds and hundreds of companies over that time. And, you know, the same thing has been coming back over and over. They’re so ready to get started, to your point in the question you raised, the IT department, the Innovation Department, the c-suite and the workers themselves are seeing real time, real term value because this heads-up, hands free capability that the HoloLens is brought to their workforce is solving problems right now. An example of the feedback that we received over the past few years that has really continued to shift our direction to where you see us now: we heard over and over, “hey, there’s nothing like this for providing real time remote expert assistance when someone is somewhere difficult to go, they don’t have the right expert on the ground. And this technology enables us to bring an expert from anywhere in the world and not only see through someone else’s eyes but draw right into their world.”
Lorraine Bardeen: 03:54 So that was one big cluster feedback that came back. And out of that we’ve delivered in APP to market called a Dynamic 365 Remote Assist. Oh, another cluster around, “hey, there’s nothing like this type of technology, the HoloLens, for enabling us to lay out a three-dimensional objects in space at real world size and scale and make the kind of decisions we always had to have a physical model to make before.” That was another whole cluster we’ve delivered, a product called Dynamics 365 layout into market also in October. Then there’s this whole area of feedback and innovation that’s been happening over the past couple of years and helping a first-line workforce. These are people who move around the real world, use their hands to do their jobs and can only use technology if it’s heads up and hands free while helping them learn, helping them upskill, reskill respond to the incredible changes across so many industries and for that challenge and opportunity, we’ve just recently announced a new product Dynamics 365 Guides, which is in public preview now.
We can talk more about those products, but overall, the answer to your question is over these multiple years, we paid attention to what customers are looking for and they’re looking for not only the ability to get started right away, but frankly most of them want out of the box value. So as they develop their own ideas, their own applications, they start to use mixed reality services you’ve talked about to build their own specialized apps. Meanwhile, they want to be using this product right away. And so they’re really looking for apps that they can, they can spin up, start using day one and get value for their company. So that’s the focus as we’ve shifted it. Do you want me to switch into your second part of your question and talk about why Dynamics makes such a good home for these products?
Mary Jo Foley: 05:37 Definitely. Because like I said, I get that question all the time because people, when they think of Dynamics, they think of ERP and CRM and then you’re like, and there’s also this mixed reality part of it. Wait, what? You know.
Lorraine Bardeen: 05:50 Yeah, no, absolutely. It’s been such a good home for our applications and our team, frankly, over the past year and here’s how I think about it. So Dynamics 365 from Microsoft represents a very rapidly growing business application space. And this is a space that Microsoft has played a big part in for a long time. And then over the past couple of years we’ve been growing the Power Platform, which I know you, you’re quite an expert on. And the Power Platform includes things like Power BI for sort of a citizen developer ability to analyze data rapidly and then share across your whole organization.
Lorraine Bardeen: 06:29 It also includes Power Apps, which enables anyone to make applications and then includes the very powerful common data service. And I say all of this because there’s two main answers to your question about why we belong in Dynamics. One answer is this Power Platform itself. If you think about from an IT decision makers standpoint, if you’re going to deploy something like mixed reality into your organization, maybe you’re completely convinced that you’re going to see near term ROI in making this decision, but you’re still a little bit nervous about, wow, how am I going to hook this into all of my other IT infrastructure?
Well, I really believe that part of the answer is in this Power Platform because all of our products either currently sit on top of and are integrated with CDS or they’re rapidly on their path.
Lorraine Bardeen: 07:15 And what that means is even if you don’t use it a Dynamics 365 product right now, and you don’t have currently have a plan to, as long as you’re using a common data schema that’s published to GitHub, that really helps to maneuver data into a way that’s useful across departments, then you can just integrate Remote Assist, Layout Guides, etc, right into your existing workflows and really treat these mixed reality applications as an extension of your other departmental apps, whether they’re created by Microsoft or not created by Microsoft.
And so one example we’ve shown and that companies are using is because of the CDS integration, we show Dynamics 365 Remote Assist integrated on the front end with the field service application, which means that a field service engineer can get all their scheduling and routing, preparation, etc from the field service mobile app and then jump right into Remote Assist. All the relevant data transfers over work order etc, without having sort of any of these cliffs between apps. So that’s one major reason. Frankly, the power of the power of platform is such a good fit for applications.
Lorraine Bardeen: 08:18 And the second main reason is the way that the Dynamics 365 product group thinks about its offerings is very much from a company’s point of view. We think about what are the challenges and opportunities that a company has around its product development, around its operations, around the people, the talent that makeup that organization and around its customers.
And you might’ve seen this, this digital feedback loop visually laid out, but that’s exactly how we think about delivering value through mixed reality applications, helping with product development, helping with customer service, helping with talent development, like first-line workers, helping with the challenges of operating in an increasingly complex environment.
Lorraine Bardeen: 09:04 So we had such a close alignment in perspective in approaches to serving customers and then frankly, being able to use the Power Platform to give a really seamless experience. Overall, those are the reasons that really resonated for me and it’s been such a good fit.
Mary Jo Foley: 09:20 That’s the best explanation I’ve heard of the Power Platform. And the reason I laughed when you brought it up is I’ve been joking on Twitter with people that you guys need a Power Platform onesie for this HoloLens. Yeah, you bring up a really interesting point because I think when people, again, when they hear Dynamics, they think automatically like, oh, so I have to be a Dynamics user already to now use these mixed reality Apps, right? So I have to use Microsoft CRM or Microsoft ERP, but that’s not actually the case. Right? You don’t have to be using those.
Lorraine Bardeen: 09:58 No, absolutely not. And you know, if a company is ready to consider it, we’re always here and eager to help, but you absolutely do not. A lot of the Dynamics 365 portfolio of applications that you can just pick up and start using it in five minutes and just get a sense of the value to the organization. That includes all of the new artificial intelligence supported applications and includes all of the mixed reality applications as well. Plus many, many others. You know, for example, to use Power BI, which is part of the Power Platform, you don’t need to use any of the product either. What we’re all taking a shared commitment to using the common data schema as part of the common data model. And many of us are prioritizing use of the common data service throughout.
Lorraine Bardeen: 10:43 That is such a critical part. You know, just as a leader of a APP engineering team, I’ll tell you, as we moved into the group and I caught up, it had been a few months since I caught up on where we were with common data service. I was so delighted because I was able to really rapidly accelerate a number of our APP development plans. Things that we didn’t have to do because they’re already provided as part of the Power Platform. So that’s why I’m such a crazy advocate for it because it’s really helped me as an app developer inside of Microsoft.
Mary Jo Foley: 11:16 Oh wow. Interesting. Okay. So a couple of questions from Petri readers that kind of fit in here. One is from Greg Alto. He said, ”Why do I need mixed reality for an ERP solution versus mixed reality for a phone, tablet or laptop?” So again, I think he, like many, are thinking I need to start out with Dynamics 365 apps and instead of just taking a mixed reality APP and putting it on a phone, tablet or a laptop, again, that’s not the case.
Lorraine Bardeen: 11:45 Yeah, not the case. He might, I don’t know, I might be reading too much into what you said, but he also be asking, why would I need to use a HoloLens versus a phone? Do you think he’s asking that or not?
Mary Jo Foley: 11:59 Yeah. Well. So he, I think he may also be asking that as, as is another reader, factoryoptimizr who said, I’m wondering if Microsoft has plans to apply this technology, meaning your Dynamics 365 mixed reality apps, to non-HoloLens mobile devices like smartphones and tablets.
Lorraine Bardeen: 12:19 Gotcha. Okay. So I’ll wrap those two together. So one thing I would say is we, to a certain degree, we answered the question that it’s, it’s certainly not necessary to start with an ERP solution and there are certain scenarios where I think mixed reality is gonna play a huge part in ERP. I’ll give a few tantalizing suggestions and I’m sure you and I will be following up on over time. One of them is in warehouse management. You can just, you can easily see that as the first-line workers we’ve been serving with Remote Assist with Layout, now with Guides.
There’s a huge amount of complexity in warehouse management and that is a core part of many ERP solutions. So that’s an area where I expect we’ll see more and more innovation. Then also just in enterprise asset management, that’s a core part of many ERP solutions and really understanding these complex high, high value assets and equipment and how they move through their product life cycle, including through repair and maintenance and all of that is something where mixed reality can play a big part.
Lorraine Bardeen: 13:16 So that’s one part of the question. And then talking about, phone, tablet, laptop, et cetera. Remote Assistant layout, which were in-market were released first for HoloLens. Guides is released for HoloLens and then all of these will be available on HoloLens 2 as soon as it starts shipping. That said, we also have been, like I said, listening, learning, paying close attention and collaborating with customers.
There are definitely cases where a phone or a tablet could be quite helpful. Now the one thing I want to establish though is that the vast majority of our customers who are using our Dynamics 365 mixed reality applications are first-line workers. So, their companies or the decision makers and the first-line workers are the people who are using the devices, who are using HoloLens and using the apps.
Lorraine Bardeen: 14:04 And as a reminder, I remember I said heads up, hands free before that is a really critical part. So many of these jobs you have to hold tools or you have to push a cart or you have to be able to pick things up and put them down and it’s actually a hassle to be holding a phone and trying to figure out what to do with it. So many, many, many of these scenarios are heads up, hands free only, and you’ll find that they’re going to roll out entirely and dominantly using HoloLens and HoloLens 2.
Now there are some cases though where we heard consistently that would be really valuable to have an option on mobile phone. And so we have now announced that we’re extending support or in some cases starting with mobile phone support for some scenarios. So two announcements a couple of weeks ago. One, we announced that we’re extending Remote Assist to mobile phones starting with Android.
Lorraine Bardeen: 14:53 And then following up later on with iOS. And starting with Android because our customer research and time with customers indicated that there are more for some workers who are using Android devices. Really quickly the scenario here is, you’re a field service worker or someone working in manufacturing and maybe there’s a inspection round that needs to happen frequently and the person who is certified to do that inspection can’t always be there onsite is easily or efficiently as they’d like. And so you can partner with them, collaborate with them to give them a view into the inspection that needs to happen. They can sign off on this spectrum remotely while you hold up the phone.
They can see through the phone using Remote Assist, do some annotation, etc, and then move on and sign off. That said, if there’s any sort of complex repair that’s going to require tools and two hands, which many do, that’s when it’s time to switch to HoloLens.
Lorraine Bardeen: 15:46 So that’s one of the announcements we made and that’s a case where we think a mobile phone is the better solution. [Two,] we also announced a new product called Dynamics 365 Product Visualize. This is starting with a phone only and then we’ll expand over time. And this is available on iOS, phone and tablet. What Product Visualize enables you to do is share with your customer a full size, at scale 3-D view of a product that you can see in your space, maybe in a manufacturing floor, in a retail store, etc.
And really use that to make a decision about the product, in a way that you couldn’t perhaps, from seeing the product on screen. And so we’re bringing that to a mobile first and then we’ll follow up with support for further end points.
Mary Jo Foley: 16:32 Okay. I was wondering if you were going to bring that to HoloLens at some point when you announced it a couple of weeks ago.
Lorraine Bardeen: 16:37 I think we will. We’re just listening and learning just like we have been with everything else, but I would absolutely expect we will.
Mary Jo Foley: 16:43 Okay. Now here’s, here’s a question. I don’t know if it’s too early to answer or not, but again, factoryoptimizr asks if Microsoft has any plans to componentize or open source, the remote support technology that you’ve demo-ed to make it available to more developers?
Lorraine Bardeen: 17:01 Yeah, this is a great question. So right now, so Remote Assist uses teams, to provide this sort of communication infrastructure and layer. And so we have a very close partnership with the Teams team, which I love saying. And work together to make sure that we can provide this product around the world, wherever people use HoloLens. And then, now Remote Assist phones. So in terms of componentizing, here’s what we’ve done so far.
Lorraine Bardeen: 17:28 Right now Remote Assist can be called from another application. If the person asking the question is building mixed reality applications, we have a pretty simple protocol that enables them to basically embed calling Remote Assist from their application. And that’s possible right now. But, I don’t think it’s what they’re really asking a fully componentized open source solution.
We don’t have a plan to do that right now, but I will say another, another thing that we’re thinking through, you probably got the impression that I’m a big fan of power apps and so we’re really thinking through how can we collaborate closely with the team behind power apps and bring more and more functionality to that space over time.
Mary Jo Foley: 18:11 Okay. Yep. That makes a lot of sense. Given the dependencies well not dependencies, but you’re betting on the Power Platform, right? (Lorraine: Absolutely.) I saw a lot of examples, case studies, demos during the HoloLens 2 launch about here’s how people could use this, which I think is really useful because otherwise it’s pie in the sky stuff. But Tina asked in our forums, which companies Microsoft and the Dynamics 365 team are working with, who could share practical APP models for the rest of us.
Lorraine Bardeen: 18:50 Yeah. So I can answer the question as far as I’m interpreting it at least when she says practical APP models. I wonder if she’s curious about whether some of them are sort of extending or whether she’s curious about their extended workflows before and after the APP. But we can maybe answer that another time. So let me start, so Remote Assist and layout have both been in market since October. So we have many customers who are using those products both before we formally sort of jade them and after. So for example, Chevron’s a great example who has been using Remote Assist extensively around the world with many, many different engineers.
Lorraine Bardeen: 19:34 And Chevron is a core example of not only heads-up, hands-free is the only acceptable path. And we’ve heard stories about how they’ve tried to solve heads up, hands free in the past. It’s been very creative, I’ll tell you. But something that, something that they’re able to do much more effectively using a HoloLens and letting each engineer have access to HoloLens whenever they need one, so they can get help where they need it.
And we’ve heard so many stories, like when you run a complex global operation, no matter what industry you’re in, it’s impossible to know where you’re going to need what help and you know, one of my favorite quotes about Remote Assist of all time is Ed Moore, the head of innovation at Chevron, who was behind the original analysis and now of course is deploying through the IT infrastructure. He says, I can put an engineer anywhere in the world in under a minute. And that’s the way he sees the product. And that’s a great, it really underlies why they’re investing so significantly moving forward so rapidly.
Lorraine Bardeen: 20:32 And that’s true for a lot of companies. You know another company that has been really instrumental in defining the product itself for layout is ZF, which is a German car parts manufacturer. They’ve been just incredible in helping us actually hone in on what are the highest value use cases for layout, a short-term, mid-term and long-term. Because if you think about Layout, it can do a lot.
It enables you to bring in any and all 3-D objects in your company’s existence and lay them out in space and really decide what’s the best layout for manufacturing line, for a retail store, for a restaurant, shop, etc. What they’ve also helped us discover is some really specific use cases that are so simple, but so powerful. For example, just simply fit testing.
Lorraine Bardeen: 21:26 It turns out that in manufacturing floors there is a ton of fit testing that needs to be done every single day. Just to make sure that our people, you know, safety is usually the top priority for from a factory, that people can move through the space safely. That equipment can move to this space safely and vehicles in many cases. And, so, just doing a quick check, can we still fit what we need to fit through the space, while things have changed in the factory over the past 24 hours? Just that simple scenario alone was a really insightful for us.
And, and ZF. It’s been a great partner and they’re also deploying Layout as well. And then Guides is our newest announced product. We have a very healthy set of preview customers who’ve been giving us a ton of feedback and this is one that I think might be interesting to your listeners over time that Guides is both being used by customers who are building their own Guides in house, companies who are building their own Guides in house and then deploying to their workers. But it’s also being used by curriculum providers and companies who deliver training to other companies.
So for example, Crane Morley, which is a performance improvement firm. They’ve been using Guides to create mixed reality curriculum for Mercedes, for their technicians to learn how to service and maintain new models. And they’ve been very, very happy with the ability to use Guides there. In preview of course. There’s a number of different customers in preview who are, who are moving very quickly with Guides.
Mary Jo Foley: 22:55 Hmm. I saw, I think it was you who tweeted it. Somebody tweeted just kind of a funny example of Guides that with somebody repairing, how to repair a coffee machine in your office.
Lorraine Bardeen: 23:05 That’s exactly right.
Mary Jo Foley: 23:07 But you know what, I thought that actually is good for people to see because I think some of these things seem so lofty, you know, like Chevron and these kinds of companies, but then you’re like, wait, I could do something like that. I could put something up for my customers about here’s how you fix some piece of equipment and they could actually do it themselves.
Lorraine Bardeen: 23:26 You’re absolutely right. One of the preview customers had a hackathon where all the employees just came together and made Guides, tons and tons of different types of Guides. And, it’s fun. You know, people feel really creative. It’s like a lot like making a PowerPoint slide when you’re putting a guide together or sorry, a PowerPoint presentation. We really focused on making it as easy as possible to author. And so some other fun examples. One of the Preview customers, one of the energetic employees made a CPR guide.
Rather than having to go and use a printed manual and then we’ve had employees take their HoloLens home and make cooking Guides. What happens a lot when you work at Microsoft as you become the extended IT support for everyone you’ve ever met. We’ve had a few people, make Guides on setting up a home network or you know, debugging your Wi-Fi. So it, it is an accessible product that I think people are having fun with. And I agree with you. It’s fun to see the results of that.
Mary Jo Foley: 24:28 Okay. Now I’ve got, this is probably the heaviest question we’ve got from one of our readers, but I understand where he’s coming from. This is Elle Sobrato and he said, I’m an architect who seen Microsoft reboot its UI technology over and over, burning early adopters. And he cites as examples, Silverlight, Win RT, weird UI, I don’t know what that one is, etc. Win RT, weird UI. I see what he’s saying – the UI for Win RT. Okay. He’s asking basically for guarantees, which I know it’s impossible to make, but he’s saying what commitments is your team making this time that you didn’t before to ensure that I’m not going to have to rewrite yet another Microsoft platform in two years or worse, for iOS and Android when their AR glasses enter the market.
So I mean, you can’t guarantee somebody, hey, it’s never going to change. But I understand his point too. It’s like things have changed a lot in certain parts of Microsoft and it’s a big bet when you bet on something like mixed reality or HoloLens. So what, what kind of guarantees can you give him that you as Microsoft are dedicated to the platform?
Lorraine Bardeen: 25:37 Absolutely. So, I mean, first I would say the best guarantee is me and my team. I mean, we’re like him we’re APP developers. So I partner very, very closely with the team that makes both the hardware device and the operating system and tools layers. My team builds applications on top of those UI controls on top of that operating system on top of the UX guidance, etc. And so, you know, at our best, my team should be the strongest internal advocate for the person who asked that question. And for everyone else that you could possibly have. We’re doing everything we can to make sure that we both drive forward with as much innovation to serve customers and delight developers for sure as possible while maintaining that level of predictability and consistency that developers need to make a business frankly, and that customers need to trust to deploy into their operation.
Lorraine Bardeen: 26:32 So that’s our balance to strike is advocates for the broader developer community. And then specifically I think interesting thing for that person who asked the question to look forward to is as we really evolve and learn in this mixed reality space, I would expect we are going to continue to see changes in natural user interaction. There’s no way that that’s not going to continue to improve.
And so this is something that we think about a lot. You know, we developed our applications for HoloLens using air tap and the gestures that were really, really solid and really, really strong for UI controls. And now that there’s more understanding of how hands move that’s built into HoloLens 2, we’re taking advantage of that and starting to develop some additional functionality for applications. So the way we see that is yes, we do have to shift a little bit how we think about the user interaction. But it’s a great opportunity to continue to make that more and more and more natural for our customers. And as we think, as long as we’re communicating really openly, as long as we’re making any sort of user interaction transition as seamless as possible, hopefully our customers will be the ones who benefit.
Mary Jo Foley: 27:50 Okay. Great. I have one final question for you before we wrap it up. When, when someone says to you, okay, I’m an IT Pro, I’m interested in this, what can I do now to start getting ready for mixed reality? What’s your elevator pitch for them?
Lorraine Bardeen: 28:05 Oh, of course. I would love them to try it hands on in the best way. Most people really, really learn. I would love them to get access to a HoloLens and just try all the applications, ask questions, give feedback if it’s not working for them, but I’d love them to just get going. Now if they don’t have access to HoloLens for some reason and or they don’t think they will I’m really happy that we’re making some of this functionality available on mobile phones as well. So they can get started with Remote Assist if they have Android devices.
Then get started with Product Visualize on iOS devices and we’ll keep an eye on where and how we should extend functionality more in the future. But the bottom line to me is this is a space where we’re moving into computing and that really understands and looks from the point of view of the person. The worker, the leader, etc. So the best way to decide is to try it yourself, to get it into operation to see if you’re, if you’re a business decision maker, if you’re saving money, empowering workers, improving quality in your operation, the results should speak for themselves. And that’s really the best way to get going.
Mary Jo Foley: 29:18 Hmm. Great. Well, thank you so much Lorraine for the great answers and useful dialogue on this topic and thank you to all our listeners too, for tuning in today to today’s MJFChat, which was all about Microsoft’s mixed reality for business strategy. We’ll be back again in a couple of weeks with our next guest, so make sure to watch for that. And when I post who it is, send in your questions using the Petri.com MJFChat forum area. Look there too for the audio recording and the transcript of this and all our chats. Thank you very much.
- This reply was modified 4 months ago by Brad Sams.
- This reply was modified 4 months ago by Brad Sams.
- This reply was modified 4 months ago by Mary Jo Foley.
- This reply was modified 4 months ago by Brad Sams.
- This reply was modified 4 months ago by Brad Sams.
- This reply was modified 4 months ago by Brad Sams.
- This reply was modified 4 months ago by Brad Sams.