Thursday, February 9, 2017

Xamarin Forms and AI Bot Framework with LUIS - Part 1


Hi Guys, We are about to discuss another Xamarin.Forms article which introduce the idea of conversational computing in an organic way (AI), where hoomans can use their natural language in making a command to apps which can have an IoT integration or can be simple conversational bot with a human understanding.

First, we gonna discuss about Language Understanding Intelligent Services (LUIS) and how to utilize its capability. This is a three part series and if you already confident about the topic you can proceed to the next.

Xamarin.Forms and AI Bot Framework with LUIS -  Part 2

Xamarin.Forms and AI Bot Framework with LUIS -  Part 3

Benefits to using LUIS

One of the key problems in human-computer interactions is the ability of the computer to understand what a person wants, and to find the pieces of information that are relevant to their intent. For example, in a travel agent app, you might say "Book me a ticket to Paris" in which case there is the intention to "BookTicket" while "Paris" is the location entity. Intention can be defined as the desired action and usually contains a verb, in this case "book", and the entity is the topic/subject, in this case "Paris", of the action.

Using LUIS for the first time

Language Understanding Intelligent Services (LUIS) brings the power of machine learning to your apps, you can custom design a set of intentions and entities that are relevant to your application, then let LUIS guide you through the process of building a language understanding system.

To use LUIS, first make sure that you have an up-to-date version of Microsoft Edge, Internet Explorer or Google Chrome. Go to the home page and log in. You will see a screen like the one below.

Creating your first LUIS application

LUIS applications are centered around a domain-specific topic, for example booking of tickets, flights, hotels, rental cars etc. or content related to exercising, tracking fitness efforts and setting goals. You need to decide on a domain-specific topic before you can create your LUIS application. In this case, let's take the example of a virtual travel booking agency application. In the application, you will bundle together the intents and entities that are important to your task. Two intents relevant to the domain of travel booking are "BookFlight" and "GetWeather". Two entities that are important are "Location" and "DateTime". Once you have identified the intents and entities, it is possible for LUIS to take appropriate action, when a user interacts with your application.

Step 1: Creating an application

Step 2: Adding intents, entities and labels

Next, we will add two intents to the application. At the top left of the menu panel, you will see an area for intents. All applications come with one pre-defined intent, None. This will recognize user statements that have nothing to do with the application, for example if someone says "Get me a great cookie recipe".

Go ahead and click + next to Intents on the horizontal bar. You'll see a dialog box appear to add a new intent. Enter the intent name of "BookFlight", and the example command that triggers the intent as "Book flight to Paris". This will look like the screenshot below.

Click Save, and the utterance will be presented for labeling. The intent "BookFlight" (just click on it) will be highlighted, and you will see a drop-down with the entities you've defined. Click Submit to submit the utterance to your LUIS app.

Defining entities

  1. In the Application Editor workspace, find Entities in the left-hand menu panel, then click the + sign.
  2. In the Add a new Entity dialog box, type "Location" as the entity name.
  3. Click the plus sign next to the Entity Children.
  4. In the text box that appears, type the name of the first child, "ToLocation".
  5. Click the plus sign again to add the second child, “FromLocation”, and so on.
  6. To delete a child, if you made a mistake, click the trash can sign next to the entity child.
  7. When finished, click "Save".

Step 3: Seeding the system by labeling utterances

With a set of intents and entities defined, the next step is to provide more examples of utterances that illustrate these concepts. Click on the New Utterances tab at the top of the screen. Type "Book a flight to London" into the entry box and hit Enter. You will see a drop-down box showing the possible intents. Select "BookFlight" by highlighting it. Click on "london" and select "Location" from the drop-down box and you'll see the word "london" highlighted in yellow, indicating that you've labeled the word "london" as a "Location". Choose whether it is a "ToLocation" or "FromLocation", then click Submit to submit this label.

The system needs to be seeded with several examples of each intent, and several examples of each entity. Don't forget to add an example or two of a None intent, for example, enter "I like ice cream". Note, that LUIS converts all utterances to lower case.

The system has now been seeded with enough data to deploy an initial application. That is done by training and publishing a model.

Step 4: Training

When you "train" a model, LUIS generalizes from the examples you have labeled, and develops code to recognize the relevant intents and entities in the future. Internally, LUIS uses logistic regression classifiers to determine intents, and conditional random fields (CRFs) to determine the entities. The training process results in optimized classifiers and CRFs, referred to as models, that LUIS can use in the future. To do training, just click the Train button at the left bottom corner of the page. Training also occurs automatically with regular intervals.

Step 5: Publishing a model

The next step is to deploy the model to an HTTP endpoint that will interpret the sentences we send it. Click the Publish button in the upper left-hand corner, and then Publish web service in the resulting window. After a couple of moments, you will see a url that makes your model available as a web service. LUIS will look like the below screenshot. To acquire a subscription key, see Creating Subscription Keys Via Azure Ibiza.

Understanding the JSON response

  1. Set the URL parameter to be your query/utterance, for example, "Book me a flight to Boston on May 4", then click Update published application..
  2. Hit the Enter key or click on the generated URL in the dialog box.

FieldJson typeDescription of content in example
QuerystringQuery/utterance: "Book me a flight to Boston on May 4"
IntentsobjectAll intents listed (each with their highest score): "BookFlight", "None", "GetWeather"
ScorenumberConfidence score, between 0 and 1. Only the highest score is listed for each intent.
Entitiesobject"boston", Type: Location::ToLocation (parent::child). "may 4", Type:
ScorenumberConfidence score, between 0 and 1. Only the highest score is listed for each entity.

Oh right, we are good but we are not done yet

On our part 2 we gonna continue by creating our Bot Connector using Microsoft Bot Framework,


You have now built a basic LUIS application. This application can be enhanced by adding more advanced features.

1 comment:

  1. Wow, cool post. I’d like to write like this too – taking time and real hard work to make a great article… but I put things off too much and never seem to get started. Thanks though.
    cruise ships