Mary Jo FoleyModeratorMay 20, 2019 at 11:47 am #617322
Our next MJFChat, scheduled for Monday, May 27, is between me and Gary Pretty, Conversational AI Consultant at Mando and Microsoft MVP. The general topic of our chat is how to get started with AI using Microsoft technologies like the Bot Framework, Cognitive Services and more.
There are a number of places where customers can “jump in” when they’re considering adding Microsoft AI capabilities to their apps and services. And the Microsoft Bot Framework and Cognitive Services are options for those who aren’t data scientists and/or don’t have a ready-made deep AI bench inside their organizations.
What questions do you have for Gary about the Bot Framework and Cognitive Services? I’ll be chatting with him on May 27 and will ask some of your best questions directly to him. Just add your questions below and maybe you’ll be mentioned during our next audio chat.
BotsorNotsParticipantMay 26, 2019 at 3:16 pm #617503
Hi Mary Jo,
We heard a lot about bots a few years ago at build but now the conversation has gone a bit quiet. Can you explain if bots really are Microsoft’s focus anymore or is this a thing they just built, tossed out into the open to see what happens?
Brad SamsKeymasterMay 30, 2019 at 12:59 pm #617613
You can find the audio playback, here.
Mary Jo Foley: 00:01
Hi, you’re listening to Petri.com’s MJF chat show. I Am Mary Jo Foley, AKA your Petri.com community magnet. I’m here to interview industry experts about various topics that you, our readers want to know about. So today’s MJFChat is all about how tech pros can get started with AI using Microsoft technologies. My guest today is Gary Pretty Conversational AI Consultant at Mando and a Microsoft MVP.
Welcome Gary. It’s so great to have you on. We’re longtime friends and colleagues, so this is going to be really fun. There’s a lot going on right now in the Microsoft AI and conversation spaces. I was just at build where Microsoft showed off some new plans for their conversational engine that they’re going to bake into Cortana and make available to others via the Bot framework and Azure services. Cognitive services are all over the map. They’re springing up, it feels like almost daily. I feel like it’s a very hot topic right now. But I will say in talking to some tech pros, I know more than a few feel a little overwhelmed and under prepared to take advantage of these kinds of technologies. So here’s looking at you, Gary, I want you to help explain why they don’t need to be afraid and why they should be excited about jumping in.
Gary Pretty: 01:36
Sure.l guess the first thing to say in terms of not being afraid to jump in and why you should be excited, is this technology really allows you to get a lot done and have a real impact with very little effort. This is a technology that’s only been around for say like two and a half years now, since built two or three years ago. When it was first introduced, the idea behind it was to give developers the ability to build one single Bot application, maintain a single Bot application, but then make that available on multiple channels.
So things like Skype, maybe web chat, Facebook messenger, and the idea of bots really isn’t anything new, but up until fairly recently, we would need to go and build an individual block for each one of those channels. But I guess the first thing to say really is behind the scenes of Bot is just a web application. It’s a web application and the Bot builder SDK, which we use to build Bots really just handles all the complicated bits for you. So sending messages to and from the channels. So receiving a message from Facebook and sending messages back to a user. All of that complexity is actually wrapped up for you in the Bot framework connector service that Microsoft handles, which is fantastic. And then you can focus as a developer on simply building a great conversation. You know, that’s the first thing I would, I always do encourage people to think about when they started to think, should I be getting involved with bot building.
Mary Jo Foley: 03:24
That’s a really good beginning. We also had a question from a reader calling himself or herself “botsornot” saying we heard a lot about bots from Microsoft a few years ago at Build as you just alluded to. But the conversation feels like it’s gone a bit quiet. Can you explain if bots are really Microsoft’s focus anymore or if this is just a thing they built, tossed out into the open to see what happens?
Gary Pretty: 03:53
It’s a good question. I think we all know that sometimes technologies get thrown against the wall to see what sticks. I think the message I’ve certainly received from Microsoft over the last couple of years is that they are really betting big on this sort of technology. The Bot builder SDK and the Bot framework really are at the core of the AI services.
They are core pillar of the AI services that they want to offer for developers. I certainly don’t get any sense that they are just throwing it to see what happens. I think we’ve seen the growth. There’s been a fairly rapid growth of the uptake of the technology. There are a lot of Bots out there that are built on this platform. There’s a very good chance that, you know, on any given day you’ll be interacting with one or more bots or services that are using this SDK under the hood. But ultimately as we’ve talked about before, if you’re a developer that needs the bot framework that you want to build with the bot framework. You know about it, it’s there. It’s one of the most important tools in your toolkit.
But just like so many of those other, you know, products and services on Azure and elsewhere in the technology industry, sometimes it just takes somebody to introduce you to them, for you to actually pick it up and get going. Just because you maybe never heard of it or you don’t know how to get started doesn’t mean it’s difficult. There are so many products and technologies out there right now that it’s really hard for some of them to make a big noise over some of the bigger projects that everybody is using. You know, maybe like dot net core for example. But no, this is a super hot area right now and Microsoft is investing big in this space.
Mary Jo Foley: 05:45
Do you think it’s a good strategy for someone’s that has never really dabbled much with AI to start with the Bot framework or is that kind of more for someone who’s a little further advanced?
Gary Pretty: 05:59
No, I think it’s a really good idea. Actually, I did that exact thing myself. So I’d never dabbled in anything to do with AI at all. I’d never used any of Microsoft cognitive services and I saw the Bot framework at Build and thought, Hey, that’s pretty neat. I think I can build something pretty cool with that for some of our customers. I went away and built a proof of concept in a few hours and that’s really straightforward and simple to do.
It really allows you to dip your toe in the water because as we’ll come on to talk about later, the other services that you use with the Bot builder SDK, the cognitive services, the things that really are the magic to your Bot are really then going to take you further down that journey into using those AI services that Microsoft have to offer. And you can keep going and using things like as your machine learning, etc. But the Bot Builder SDK is a great place to start. It’s a phenomenally powerful tool for building awesome solutions for either yourself or your users or your customers. It really is awesome.
Mary Jo Foley: 07:08
Do you feel like somebody needs to understand concepts around data science or machine learning or do you even need to be a programmer to start using the Bot Framework? Like what, where do you need to come into the equation?
Gary Pretty: 07:21
Okay. So you definitely don’t need to understand things like machine learning and any, you know, deep complex AI topics. It’s worth calling out here that I’m a Microsoft AI MVP, but I focus on the Bot builder SDK, the Bot framework and the cognitive services. I don’t write my own machine learning algorithms. I don’t really get that deep. I am using all the services that are available off the shelf to combine them to build awesome solutions.
To get started really easily. You can go into the Azure portal and spin up a Bot app in just a couple of minutes. And you can say, I want a Bot app and you can pick from some templates. You can either have a really basic echo Bot, which will just repeat back to you what you’ve said in the web chat window. And that can be really useful to just get used how am I sending and receiving messages? Then you can spin up another web app, which is then starting to introduce some of those cognitive services that we talked about and some of the key concepts there. Then once you have progressed and you understand all of those concepts, Microsoft is then doing an amazing job of saying, okay, we don’t want our customers to reinvent the wheel.
One of the things that I think is happening across Microsoft, certainly over the last 12 to 18 months is they’re saying they are looking at their customers and they’re realizing that everybody was actually reinventing the same thing over and over again. And a really good example of this was the Virtual Assistants Accelerator. So the Virtual Assistant Accelerator is a solution, a prebuilt solution that’s open source or made available to developers by Microsoft. It’s opening GitHub and it is essentially a virtual assistant built on the Bot framework, which will allow a user who is interacting with it maybe by a web chat or via Microsoft teams, which is one of the other channels to do things like book appointments in their calendar, but using conversations so they can say, “book me a meeting with Mary Jo on Wednesday between two and four in meeting room one” and they’re going to use the cognitive services and the Bot framework to make that happen.
But you don’t have to build that from scratch. They’re going to give you the base code for you to then tweak and hone to make it your own. And because they realize that every enterprise customer that they were talking to was looking to try and do the same thing. So building out the solutions and actually sharing the best practices that they’re finding means even if you’re looking to build out something really quite complicated, it doesn’t have to take a lot of time or a lot of effort. You can really focus on tailoring it for your needs.
Mary Jo Foley: 11:37
That sounds also like a good way for somebody who isn’t a developer at all, but is more like a tech pro could start checking this technology out and seeing how it can be used in your organization.
Gary Pretty: 11:48
Yes, absolutely you can. As a tech pro, as long as you’ve got some basic skills, you know, the ability to maybe go and check out a repo from GitHub and get a project up and running and see it working. That’s one really good way of doing it. I mean, in other ways, just to go and check out some of the awesome content that’s available online around this area.
I’m one of the AI MVPS, but I’ve got colleagues of mine who produce some phenomenal video content on youtube where you can just go and watch five-minute videos. They will take you from zero to hero in terms of Bots and show you how do these things work? And they’ll explain those core concepts for you. The community is also really, really friendly. You know, the idea that anybody can really reach out to any one of us and we’ll do our absolute best to help. It’s a very unusual week if I don’t get two or three requests just by a DMs on Twitter, just saying, I’m just getting started. I’ve had a problem. How can I fix this?
And I quite often work with people backwards and forwards until they’ve got their solution working because that’s where I really see and get my value from being an MVP is showing the people how they can work with this technology and seeing them be successful.
Mary Jo Foley: 13:10
So we, we’ve mentioned the word cognitive services a couple of times, but can you get a little deeper on that? When I talk about them in articles, I just say these are APIs that developers can add to their products and services to make them AI enabled. But that’s kind of a little simplistic, I admit. But you know, if you’re a tech pro, what do you need to know about what our Cognitive Services and what kinds of things they could help you with in your job?
Gary Pretty: 13:40
Sure. I don’t necessarily think that that’s too simplistic a way to describe it. I would say exactly how I would describe it to customers that I’m talking to or other developers because the beauty of the Cognitive Services a lot of the time is their simplicity and how straight they are to pickup and use. Going back to that point that I made earlier about Microsoft wanting to not have people reinvent the wheel.
Let’s take something like image recognition. Image recognition has been around for a long time. So also things like speech recognition, but everybody was building their own models and you needed a team of data scientists. You needed you know, a lot of programmers and ultimately all different enterprises or different customers were building their own air from scratch. Now Microsoft who said, well, hang on, we’ve got a ton of experience in these areas from the last 20 to 30 years. How about we take all of the great technology that we’ve got behind the scenes and we make that available off the shelf for you to use? It’s this whole idea of democratizing AI.
Microsoft has been using that buzzword for a couple of years, but for me, it’s more than just a buzzword. It’s very much a mantra for them. The idea of getting these services and to the hands of customers so that you don’t have to have all this knowledge and you don’t have to maintain these models, so we use these cognitive services a lot with Bots.
So if you think about the Bot framework and the Bot Builder SDK as being the platform that controls sending messages between a user and a the platform, like Facebook or teams, we then use the cognitive services to understand what a user is talking about. So if they use the example for booking a meeting, on a particular day at a particular time, we can use the language understanding service to understand what their intent is, what do they mean, what are the important pieces of that phrase?
You know, it’s the time, it’s the location, it’s the attendees. Or we can use the cognitive services to our control, how we speak back to a user as well. So using things like the translation service for example, you know I think there’s around 50 different languages now that you can convert and translate between. We can actually use the translation services to detect the language that our user is speaking on the way into a Bot, convert that into a language that we understand as a developer then act on that query that the user has sent in and produce a response and then translate that back into the language and send it back to the user all in real time without any noticeable delay. I mean these services are incredibly powerful and just at Build, a couple of weeks ago we saw some really awesome services being announced, either in a preview now or private preview over the next couple of months.
Things like the personalized service that’s going to allow you to effectively get personalized results. You think about the recommendations on Amazon or on the recommendations of what to watch on Netflix. We could have built a service like that. We could have done it ourselves before that, but ultimately you would have needed a special set of skills. Right now all you need is the ability to know how to call an API. And that’s the beauty of cognitive services.
Mary Jo Foley: 17:23
Another thing we saw at Build that I thought was very interesting, but I think a lot of people didn’t quite understand what they were seeing was that turn by turn conversational engine that they showed. Where you don’t have to every time say like Cortana, blah, blah, blah, Cortana. You could just say, Cortana, what’s the weather today? Then followed up with, do I need a coat? Should I bring an umbrella?
And it will follow your conversation as if you were talking to a person. Microsoft said that’s being enabled by some of the technology they acquired when they bought Semantic Machines about a year ago. So I’m curious, we know Cortana’s not really your thing, quote-unquote, but you are a conversational AI expert and a lot arm chair industry watchers think when you say AI they think, oh, you mean Siri or Alexa or Cortana. And they just think of that as the whole kind of ball of wax there. I’m curious about that particular set of technology. We didn’t really get a sense of how close or far away that is, but what was your take on it, do you think is just demoware or do you think this is something that people like you and other people working with these technologies are going to be able to take advantage of anytime soon?
Gary Pretty: 18:42
Yeah. Okay. That’s a really good question. You know, I’ll preface this by saying this is just my opinion. I don’t have any secret sauce from Microsoft. But the one thing I would definitely say is, but we know that Microsoft has a rich history of being able to show a vision of the future. You know, I remember when Windows Mobile 6 was around and we were seeing these really visionary videos and clips of what does communication look like in 5, 10 years time?
I do think this is slightly different though. I think whilst we don’t have access to that technology right now, the services that are coming out behind things like the Bot builder SDK and Cortana, and you can build Cortana skills using the bot framework the service we’re seeing come out behind those platforms really do allow you to start having those 10 by 10 conversations. Now it may not be as advanced as what we saw in the video, right now.
But what we are seeing are things like there’s the service called QNA maker and that allows you to provide a series of questions and answers in a knowledge-base. And then when somebody asks a question, we use machine learning and it will automatically go and pick the most appropriate answer from that knowledge base. Well at Build they announced a new capability for the Q and A maker, which is followup questions and followup prompts. So you could in theory, string a whole bunch of those questions together.
You don’t need to know the order beforehand. You just need to say these questions relate to the followup questions which relates to these other questions. That’s going to allow you to do something similar to that.
Mary Jo Foley: 20:29
I think another technology you can keep an eye on from my perspective would be the project conversation letter, which right now is in the Cognitive Services labs and it’s been there for some time. But it’s still in on the active preview. Conversation Learner is, rather than you needing to go away and be a developer and build a conversation using code, where you are really controlling at quite a granular level, what does that conversation look like in terms of its structure? And we’ve still got a lot of flexibility in terms of how that conversation might branch and flow and we can make it feel supernatural.
Whereas project Conversation Learner though takes that out of the way and it learns from examples. So you as a developer or maybe even just a business user can actually train Conversation Learner to have a conversation autonomously by having a conversation with it. So you can actually provide both sides of the conversation. So you might say, Hey, is it going to rain today? And then you can provide the follow-up and that follow up can be calling out to third-party services. And what this allows you to do is with enough training, with Conversation Learner, it can intelligently switch contacts and move between different topics of a conversation fairly effortlessly.
So I think we’re a lot closer to that video than we may think. I think, you know, Microsoft’s history of showing some demoware sometimes it can feel like these things are a long way away. But looking at how far we’ve come just in the last couple of years in terms of Microsoft, they are, I think we’re going to see some of this pretty soon.
Mary Jo Foley: 22:19
To step kind of up one level, I think a lot of people want to know what kinds of things can I do better or at all with things like cognitive services and the BOT framework that might be impossible for me to do otherwise. I guess what I’m thinking about is, okay, it all, it sounds amazing. It’s like this technology is, is so futuristic, but it’s also starting to actually show up in real life. But what’s an example of something you would do with your, with one of your clients, say, um, that you couldn’t do or you couldn’t do easily without these technologies?
Gary Pretty: 22:59
Sure. So let’s put Bots aside for one second. And let’s look a really simple example with cognitive services. So we had one of our clients and they had a problem where people were visiting the website, they were searching in the search box and they were coming up with either irrelevant search results or no search results because the query that they were typing in were using different terms and different phrases that didn’t maybe match the content on the website. It was kind of a problem. So we looked elsewhere in the industry and said, okay, who does search really well? We didn’t look at Bing, we looked at Google, and we said, what does Google do that’s really awesome in terms of search.
It’s actually one of the most simple things they do is they will predict what you’re looking for, give you the answer straight away. So if I type in, what’s the exchange rate pound against the dollar, it will give me the answer. It doesn’t expect me to go and say actually the results and click through to a website. We can actually enable an experience just like that using the Cognitive Services. So what we did, we took two Cognitive Services, language understanding and the QNA maker. We train to them and it really didn’t take that long at all. I mean we’re talking a few hours for each service here and we put some questions and answers in and we trained it to understand a user’s intent.
We call those services now whenever somebody types into that search box. What that means is whenever they type into the search box as well as the searching through the standard web pages, we are also using machine learning to go and find the appropriate FAQs. So maybe they’ve typed in, I’ve lost my account number, or how can I pay by credit card? We can bring those FAQs back and display them above the search results in exactly the same way that Google does. And the reason that is so powerful is you can have a thousand different uses, say their search query in a thousand different ways, but because of the simplicity of the cognitive service, they can all resolve to the same FAQ or the same call to action.
Mary Jo Foley: 25:24
That’s very cool. Nice. Well, we’re almost out of time. Thank you so much for helping decipher these complex topics for our readers and listeners. However, before you go, I need to ask you a beer question. We are craft beer buddies. So we always talk about beer whenever we get together. My question for you today is what’s your summer go to craft beer style? I know you, you spend a lot of time in Florida, so you probably have some good options that you go to when the weather is quite warm.
Gary Pretty: 26:01
I do. I literally just got back from Florida a couple of days ago for summer vacation. My goto right now has to be the funky Buddha. The name of the brewery is actually escaping me right now, but it’s a Floridian brewery. It’s a phenomenal a beer. It’s low percentage. It’s very sessionable. So if you are in the area, and I’m sure you can probably get it around NYC as well, Mary Jo, I don’t know if you’ve had.
Mary Jo Foley: 26:41
Yeah. Funky Buddha is actually the name of the brewery, so yes. You know what’s funny, we don’t get that all that often up here, but I have had a chance to have a couple of their beers and their excellent.
Gary Pretty: 26:54
Oh yeah, that is fantastic. So definitely try and pick them up
Mary Jo Foley: 27:07
I can tell you’re, you’re turning into a Floridian, you’re picking Florida breweries. Well we’re going to be back in a couple of weeks with our next guest on MJFChat, so make sure you watch for that. I’ll be posting who it’s going to be in the forums on Petri.com And then that’ll be your signal listeners to send in your questions. Just go to the MJFChat area in the forums and in regard to this chat with Gary. Look for audio and the transcript of this as with all of our chats on the Petri.com site in the near term. Thank you very much.
- This reply was modified 3 weeks, 5 days ago by Brad Sams.
You must be logged in to reply to this topic.