Microsoft Cognitive Services offers awesome APIs and services for developers to create more intelligent applications. You can add interesting features, like people's emotions and video detection, face, speech, and vision recognition and speech and language understanding into your application.
The Emotion API takes a facial expression in an image stream as an input and returns confidence levels across a set of emotions for each face in the image like anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise, in a facial expression
Microsoft AI: Mobile Emotion Recognition Application

Where to implement 

There is no single unifying definition or theory of emotion. Nevertheless, several characteristics of an emotional reaction are frequently mentioned. Nowadays feedback is more important for product promotion and review the article, store, etc. manual entry of emotion maybe wrong so we can automate the feedback and review.

Microsoft AI: Mobile Emotion Recognition Application

In this blog, we can learn about how to implement Emotion Recognition in Xamarin.Forms with Microsoft Cognitive Services.

Getting Started with the Emotion API 

The Emotion API can detect anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise, in a facial expression. As well as returning an emotion result for a facial expression, the Emotion API also returns a bounding box for detected faces using the Face API. If a user has already called the Face API, they can submit the face rectangle as an optional input. Note that an API key must be obtained to use the Emotion API. You can follow below steps for obtained at free API key on Microsoft.com

Step 1: Click here for Generate Subscriber Key and Click on Create


Microsoft AI: Mobile Emotion Recognition Application
Step 2: You can agree the terms and Microsoft privacy statement > select the country and click on Next

Microsoft AI: Mobile Emotion Recognition Application

Step 3: Login with Microsoft /LinkedIn /Facebook /Github for create your API

Microsoft AI: Mobile Emotion Recognition Application

Step 4: You login and subscribe you should see the keys and the below information on available your subscriptions 

Microsoft AI: Mobile Emotion Recognition Application

We will use emotion API key in the following Xamarin application. before that we can test the API key

Test on Your Emotion Key

You can navigate emotion API online console application and provide the subscription key and image url . If your provide valid key and input the image, you will get it JSon data as output or relevant error

Microsoft AI: Mobile Emotion Recognition Application

Create New Xamarin Forms Application 

You can refer my previous article for setup and create new Xamarin forms application.

If you already installed VS on your machine, Open Visual Studio > Create New Xamarin Forms application like below

Microsoft AI: Mobile Emotion Recognition Application

Solution will create with all the platform and PCL project

Microsoft AI: Mobile Emotion Recognition Application

In this solution, we are going to create demo application for Article review automate based on the emotion.

Camera in Xamarin. Forms 

We can use Media.Plugin Nuget package for Capture image or select image gallery from all the mobile platform (iOS,Android,Windows)

Right Click on Solution > Type “xam.plugin” in the search box > Select all the project (PCL, iOS, Android and UWP) > Click on Install

Microsoft AI: Mobile Emotion Recognition Application

Emotion Nuget Package

 This client library is a thin C# client wrapper for the Microsoft Emotion API.The easiest way to use this client library is to get microsoft.projectoxford.emotion package from nuget .

Right Click on Solution > Type “Microsoft Emotion” in the search box > Select all the project (PCL, iOS, Android and UWP) > Click on Install

Microsoft AI: Mobile Emotion Recognition Application

UI Design 

After successfully install above two nuget package. Let start UI design from PCL project In Dotnet Standard Project > Open MainPage.Xaml page and design following xaml code


<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
xmlns:local="clr-namespace:DevEnvExeEmotion"
x:Class="DevEnvExeEmotion.MainPage">
<StackLayout Margin="20">
<StackLayout Orientation="Horizontal" Padding="30" HorizontalOptions="CenterAndExpand" >
<StackLayout x:Name="emotion">
<Label Text="Reader is :" FontSize="Large"/>
<Label x:Name="emotionResultLabel" FontSize="Large" />
</StackLayout>
<Button Text="Share Your FeedBack" Clicked="OnFeedBackClicked"></Button>
</StackLayout>
<Image x:Name="image" Source="article.png" />
<ActivityIndicator x:Name="activityIndicator" />
</StackLayout>
</ContentPage> 


In mainPage.xaml.cs file add your logic.


public partial class MainPage : ContentPage
{
EmotionServiceClient emotionClient;
MediaFile photo;
public MainPage()
{
InitializeComponent();
emotion.IsVisible = false;
emotionClient = new EmotionServiceClient("7f495be5faf643adbeead444d5b79a13");
}
For capture image, write the following code
public async Task CaptureImage()
{
await CrossMedia.Current.Initialize();
emotion.IsVisible = false;

// Take photo
if (CrossMedia.Current.IsCameraAvailable || CrossMedia.Current.IsTakePhotoSupported)
{
photo = await CrossMedia.Current.TakePhotoAsync(new StoreCameraMediaOptions
{
SaveToAlbum = false
});
}
else
{
await DisplayAlert("No Camera", "Camera unavailable.", "OK");
}
}
For Recognize emotion, write the following code
public async Task Recognizeemotion()
{
try
{
if (photo != null)
{
using (var photoStream = photo.GetStream())
{
Emotion[] emotionResult = await emotionClient.RecognizeAsync(photoStream);
if (emotionResult.Any())
{
// Emotions detected are happiness, sadness, surprise, anger, fear, contempt, disgust, or neutral.
emotionResultLabel.Text = emotionResult.FirstOrDefault().Scores.ToRankedList().FirstOrDefault().Key;
emotion.IsVisible = true;
}
photo.Dispose();
}
}
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
}

While click on Feedback button, call the above two methods

async void OnFeedBackClicked(object sender, EventArgs e)
{
//capture image
await CaptureImage();
((Button)sender).IsEnabled = false;
// Recognize emotion
await Recognizeemotion();
((Button)sender).IsEnabled = true;
}

Run the Application

Select the iOS /Android /Windows and install into the device and click on ‘Share your Feedback ‘button
Microsoft AI: Mobile Emotion Recognition Application

The device should be camera so after click on the button, capture your face for Recognize emotion.



Microsoft AI: Mobile Emotion Recognition Application

Summary


 In this article, you learned about how to generate emotion API key and implement Emotion Recognition in Xamarin.Forms with Microsoft Cognitive Services.If you have any questions/ feedback/ issues, please write in the comment box. 
Sentiment analysis seeks to understand a subject’s attitude or emotional reaction towards a specific topic or brand. Sentiment analysis does not have to be complicated and technical. It could be something as simple as getting a person in your team to find what is being said about your brand and product on review page and identify how much of it is good and how much of isn’t. There is no need for a big budget and a developer into complicated software, the cognitive service text Analytics API is a cloud-based service that provides advanced natural language processing over raw text, and includes four main functions: sentiment analysis, key phrase extraction, language detection, and entity linking.



The most of the companies and brands now use sentiment analysis to find out what people are saying about them on social media. A bad review of social media can destroy a brand’s reputation if ignored or poorly handled. They aren’t simply rating their experience with 1 star or 5 stars. They’re also expressing their thoughts, feeling, expectations in free form text. This can be challenging to handle, especially if your company is getting a lot of feedback. When you have tens or even hundreds of thousands of feedbacks to read and manage, its easy to use cognitive text analytics service API.



The Sentiment Analysis API evaluates text input and returns a sentiment score for each document, ranging from 0 (negative) to 1 (positive). This capability is useful for detecting positive and negative sentiment in social media, customer reviews, and discussion forums. Content is provided by you, models and training data are provided by the service.

Currently, Sentiment Analysis supports English, German, Spanish, and French. Other languages are in the preview. In this article, I will show how we can integrate sentiment API from Xamarin Mobile application using visual studio 2019


Text Analytics API Price

The Text Analytics API can be purchased in units of the S0-S4 tier at a fixed price. Each unit of a tier comes with included quantities of API transactions. If the user exceeds the included quantities, overages are charged at the rate specified in the pricing table below. These overages are prorated and the service is billed on a monthly basis. The included quantities in a tier are reset each month. In the S tier, the service is billed for only the amount of Text Records submitted to the service. You can read more about pricing based on the country check out here.


Create Text Analytics API Key

You need to create an Azure account, generate API key and end point URL based region for implementation to the Xamarin Mobile application.

Step 1:  Create Free Azure subscription and Login to portal
Step 2: Create On “+ Create a resource “> Under Azure Marketplace, select AI + Machine learning and discover the list of available featured. > Select “Text Analytics”



Step 3:  On the create page, provide the name, pricing, resource group and click on Create



Step 4:Wait for a few seconds. After the Cognitive Services account is successfully deployed, click the notification or tile in the dashboard to view the account information. You can copy the Endpoint URL and Key in the Overview section for API calls in our Xamarin applications.



Build Xamarin Forms Application using Visual Studio 2019

Let's start with creating a new Xamarin Forms Project using Visual Studio 2019. When accessing Visual Studio 2019 for the first time, you will come across a new interface for opening a creating the project.

Open Run >> Type “Devenev.Exe” and enter >> Create New Project (Ctrl+Shift+N) or select open recent application.



The available templates will appear on a window like below. Select Xamarin Forms application with different mobile platform.



Provide project name, Location and solution name in the following configure new project screen



Select as Blank apps and select the platform



The Solution will be created with all the platform and PCL projects.



Xamarin UI Design:

The UI will have a few elements on the screen and overlay content view window. Editor control for providing user input value and an overlay window to show the result.

You can add Newtonsoft.JSON to solutions. Right click on Solutions > Manage NuGet Packages > select Newtonsoft.Json from Browse tab > click on Install.

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
xmlns:local="clr-namespace:MobileFeedback"
:Class="MobileFeedback.MainPage">
<ContentPage.Content>
<AbsoluteLayout>
<!-- Normal Page Content -->
<StackLayout AbsoluteLayout.LayoutBounds="0, 0, 1, 1"
AbsoluteLayout.LayoutFlags="All">
<Image Source="product.gif" VerticalOptions="Start" HorizontalOptions="Start" Margin="0,0,0,0" ></Image>
<Editor x:Name="txtfeedback" WidthRequest="100" HeightRequest="200"></Editor>
<Button Text="Submit" Clicked="Submit_Clicked"></Button>
</StackLayout>
<!-- Overlay -->
<ContentView x:Name="overlay"
AbsoluteLayout.LayoutBounds="0, 0, 1, 1"
AbsoluteLayout.LayoutFlags="All"
IsVisible="False"
BackgroundColor="#C0808080"
Padding="10, 0">
<StackLayout Orientation="Vertical"
BackgroundColor="White"
HeightRequest="175"
WidthRequest="300"
HorizontalOptions="Center"
VerticalOptions="Start"
Margin="0,20,0,0" >
<Image x:Name="imgstatus" WidthRequest="70" HeightRequest="70"></Image>
<Label Text="" x:Name="lblStatus"></Label>
<StackLayout Orientation="Horizontal" HorizontalOptions="Center">
<Button Text="OK" FontSize="Small"
VerticalOptions="CenterAndExpand"
HorizontalOptions="Center"
Clicked="OnOKButtonClicked" />
</StackLayout>
</StackLayout>
</ContentView>
</AbsoluteLayout>
</ContentPage.Content>
</ContentPage>

Create Document Entity Class

Create a class for Document class, it will deserialize the response and return an object of type TextAnalyticsResponse.The response format defined by the API looks like below document entity

using System;
using System.Collections.Generic;
using System.Text;
namespace MobileFeedback
{
class Document
{
public string Id get; set; }
public double? Score get; set; }
}
class TextAnalyticsResponse
{
public List<Document> Documents get; set; }
}
}

Create SentimentAnalysisHelper Helper Class

You can replace Text Analytics API service endpoint and subscription key. If you don't already have these go back to the previous steps. Below is the complete class you need to add. We have to append /sentiment to the end of the ApiUri in order to invoke the sentiment operation.



using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Text;
namespace MobileFeedback
{
static class SentimentAnalysisHelper
{
private const string ApiUri = "<API url>”;
private const string SubscriptionKey = "<your Key>";
private const string Text = "The food was delicious and there were wonderful staff.";
private static readonly HttpClient Client = GetClient();
private static HttpClient GetClient()
{
var client = new HttpClient();
client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", SubscriptionKey);
client.DefaultRequestHeaders.Add("ContentType", "application/json");
client.DefaultRequestHeaders.Add("Accept", "application/json");
return client;
}
private static TextAnalyticsResponse DeserializeTextAnalyticsResponse(string json)
{
return JsonConvert.DeserializeObject<TextAnalyticsResponse>(json);
}
public static TextAnalyticsResponse GetSentiment(string text)
{
var body = JsonConvert.SerializeObject(new
{
Documents = new object[]
{
new
{
Text = text,
Id = Guid.NewGuid()
}
}
});
using (var content = new ByteArrayContent(Encoding.UTF8.GetBytes(body)))
{
var responseMessage = Client.PostAsync(ApiUri, content).Result;
responseMessage.EnsureSuccessStatusCode();
var json = responseMessage.Content.ReadAsStringAsync().Result;
return DeserializeTextAnalyticsResponse(json);
}
}
}
}

You can add following to code behind in the design file. we'll add a method(GetSentiment) to call the Text Analytics API sentiment endpoint. It will deserialize the response and return an object of type TextAnalyticsResponse .The method will take a string text as input, create the request body, and then send it to the Text Analytics API using an HttpClient instance.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Xamarin.Forms;
using System.Net.Http;
using System.Threading;
namespace MobileFeedback
{
public partial class MainPage : ContentPage
{
public MainPage()
{
InitializeComponent();
}
private void Submit_Clicked(object sender, EventArgs e)
{
TextAnalyticsResponse result = SentimentAnalysisHelper.GetSentiment(txtfeedback.Text);
doublescore = result.Documents.FirstOrDefault().Score;
if(score < 0.5)
{
imgstatus.Source = "bad.png";
lblStatus.Text = "This is the first time we have heard of this problem. Thank you for pointing it out to us. I assure you we will do our best to prevent it from happening again, We will contact with you regardng the issue";
}
else if(score > 0.5 && score < 0.9)
{
imgstatus.Source = "okay.png";
lblStatus.Text = "We are always eager to get feedback from our customers. Thank you for taking the time to write to us.We will improve our service and our team will contact you";
}
else if(score> 0.9)
{
imgstatus.Source = "happy.png";
lblStatus.Text = "Your valuable feedback will assist us in our continuing effort to provide our users with the best possible support experience.";
}
overlay.IsVisible = true;
}
void OnOKButtonClicked(object sender, EventArgs args)
{
overlay.IsVisible = false;
}
void OnCancelButtonClicked(object sender, EventArgs args)
{
overlay.IsVisible = false;
}
}
}

We have completed the code for consuming TextcAnalytics API. Now, we can select the platform and press F5. The output looks like below




Summary

In this article, you learned how to consume Text Analytics API and automate customer feedback without using rating. I hope this article will help you. Please leave your feedback/query using the comments box, if you like this article, please share it with your friends.
Microsoft is hosting its Future Decoded summit in Mumbai and Bengaluru on February 24 and 25, 2020 respectively, and Microsoft CEO Satya Nadella will keynote the event.

On 25 February 2020 Microsoft CEO Satya Nadella will deliver the keynote at Future Decoded, Tech Summit and share his vision for the future of technology and how Indian organizations can lead in an era of digital transformation. The keynote will be live streamed, and we invite you to participate by viewing it online. To receive the viewing link- Click here


Register to listen to Microsoft CEO Satya Nadella at Future Decoded 2020


Future Decoded Bengaluru Tech Summit - Offline View 

Did you catch Future Decoded Bengaluru Tech Summit live? If you missed it, do not fret.Let play here. #FutureDecoded

 




The Bot Framework Composer is a fantastic tool for creating/developing Microsoft bots, as it gives you the ability to sketch out and compose the whole simple and complex Bot applications within a visual interface instead of writing actual code. Composer to create the bot solution’s code and run it immediately to test it in the emulator and deploy it to Azure using the CLI. Bot developers should check out the Composer awesome tool, as this will make your Bot development even easier in terms of building smart and engaging bots with ease.

A conversation between a bot and a user often involves asking the user, for information and parsing the user's response. Bot should track the context of a conversation, so that it can manage the behavior and remember answers to previous questions. Bot Framework composer has provided different question format, in this article we are going to see how we can use the Text, Number and Datetime input from users and How we can retrieve state information from Bot state.



Prerequisites

  • Read my previous article to build and setup the bot framework Composer development environment.
  • Read, How to create Multiple Dynamic Greeting message using Bot Framework ComposerCreate New Bot Application
On the Bot Framework Composer homepage, click on New. Select Create from scratch. Click on Next. Give your bot a name. Once created, you should be able to see the following screen


Asking Questions to user

Bot Framework Composer makes it easier to collect information to the user and composer provided feature for validation and a variety of data types as follows
  1. Text Input
  2. Number Input
  3. Confirmation
  4. Multiple Choice
  5. File or attachment
  6. Date or time
  7. OAuth Login



In this article, You are going to learn more about Text, Number and Date/Time Input format and how bot can ask questions to the users. 

Composer Input Control Property – Bot Ask

  • Ask question Input dialog has 3 category options - Bot Asks, User Input and other.
  • Prompt Property -Create your custom message for Bot will send a message to the user .
  • Max turn Count – Maximum number of re- prompt attempts to collect information from user
  • Default Value - Expression to examine on each turn of the conversation as possible value to the property.
  • Allow interruptions - A boolean expression that determines whether the parent should be allowed to interrupt the input.


User Input

User Input tab options have “Property to Fill”. It’s a Property to store collected information. Input will be skipped if the property has value (unless 'Always prompt' is true). Example(user.name), user.name property can access anywhere you want to display or store. While displaying you need to use @{user.name} format.


Bot Text Input Question

The Greeting conversation activity screen, you can click on the “+” button and select the Ask question options and click on Text input .

Step 1: Add “Bot Asks” prompt of text in the text box, which you Bot required to ask questions to the user and the type of value will allow only as text .

Step 2: If you want to save user input and display somewhere else or store into the database, you can select “user Input” section and add the variable property as - user.name.

Bot Number Input Format

Bot can ask users to enter any number value using number input. To prompt a user for a number, click the + button in your screen, mouse over Ask a Question and Select the Number input.
In this following example 
  • Bot asking users to enter the no of years’ experience and provided max turn count as 1 and provided default value as 2.
  • Suppose users enter any text or any invalid character at one time, Bot will take as default value as 2.
  • In user Input options Bot will save the value “user.exp” property, it will be helpful for retrive somewhere else.


Bot Date Time Input Format

Bot Date Time Input format control which is returned as a Timex.
  • You can add the “Bot Ask” question to the Prompt text box
  • Create Property for store the datetime value, here property as “user.date”.
  • after asking for a date, the bot returns a Timex into “user.date” like below
[{

"value": "2020-02-01",

"timex": "2020-02-01"

}]

Because it's a json array with one item. So you can access the components like user.date[0].timex (or) user.date[0].value

Confirmation Message

We have asked 3 different questions to the user; now let us show confirmation message to the user with whichever user responded to the bot. You can compose messages like below 


Hello @{user.name} , You have @{user.exp} years’ experience and please join our company on @{user.date[0].value}




Test Bot Application

Next, click on “Test in Emulator” after that browser will ask you permission to open the bot emulator, once you confirmed the message, Bot emulator will open and automatically navigate to your Bot service. You can start responding to the message to the bot.

Summary

You have successfully created and run the “Ask Question bot” using Bot framework composer. I hope you have enjoyed learning. If you have any questions/feedback/issues, please write them in the comment box.
The Bot Application runs inside an application in different channel like Cortana, Skype, web chat, Facebook, Message, etc. Users can interact with bots by sending them messages, commands, and inline requests. Microsoft has announced Line App new channel added into the Azure portal.

LINE is a new communication app which allows you to make FREE voice calls and send FREE messages whenever and wherever you are. Line is available in different device iPhone, Android, Windows Phone, BlackBerry, and Nokia and even your PC and LINE is a popular messaging app with hundreds of millions of users in India, Japan, Taiwan, Thailand, Indonesia, and other countries. You can follow the below steps for Create FAQ Bot and enable your bot in the Line new channel.



Create FAQ Bot Application

You can refer to my previews article to create and build a Xamarin FAQ Bot using Azure Bot Service and deploy it into Azure. I am not using any coding for developing the Bot Application, you can follow the provided steps in the article to create and deploy FAQ Bot.

Microsoft Bot Application deploy to Line App using Azure Bot


Line Developer Portal 

We can implement a Bot Application to the Line apps. You can create a Line developer account or select an existing provider name. Create a new Line Developer App on the Page and generate an Channel Secret and Channel access token for integrating the Bot to the Line ap. You can start login or create account in the following screen.
Microsoft Bot Application deploy to Line App using Azure Bot

Create Provider

After login, start creating a provider name for your bot or if you already have the provider, select the provider name and do the setup. The provider name is individual owner or company, it’s not app name. Next Click on confirm, if you want change in the future, edit options always enable. Microsoft Bot Application deploy to Line App using Azure Bot

Line Messaging API channel 

Create a new Line App on the developer portal and generate a Channel Secret and Channel access token integrating the Bot to the page messenger. You can click on “Message API Create Channel” from the following screen.
Microsoft Bot Application deploy to Line App using Azure Bot


Create New Channel 

The following creates new channel screen provide valid your app icon.
Microsoft Bot Application deploy to Line App using Azure Bot
You can Fill out the required fields and confirm your channel settings. You can find the two developer options is available.

Developer trial - A trial plan which lets you create a bot that can send push messages and have up to 50 friends.

Free - A plan which lets you create a bot with an unlimited number of friends.The Push message cannot be sent with this plan.

Microsoft Bot Application deploy to Line App using Azure Bot

Once you've confirmed your channel settings, you'll be navigating to a following screen, which list all the apps.

Microsoft Bot Application deploy to Line App using Azure BotClick on the channel you created to access your channel settings, and scroll down to find the Basic information > Channel secret.

You can Save following somewhere for a moment for update in to Azure portal.

Channel Secret – Copy the secret code, if not available, click on Issue

Channel access Token - Scroll further to Messaging settings. There, you will see a Channel access token field, with a click on issue button forget your access token. 

Webhook URL - Webhook URL copy from Azure Portal and Update in the Line app developer portal

Microsoft Bot Application deploy to Line App using Azure Bot

Connect New LINE Channel 

Login to Azure portal > Select the “All Resources” > Select your Bot Application > Select Channels property > Find and Select New Line icon. Let us start to configure the “Line “Channel and follow the below steps, at the end of this article you will be able to deploy the Bot into the Line apps.

Microsoft Bot Application deploy to Line App using Azure Bot


The Azure Line configuration channel will generate the following Webhook URL and past/update the channel secret and access token and click Save button. You can copy following custom webhook URL and Return to the Line Developer portal and update the webhook URL

Microsoft Bot Application deploy to Line App using Azure Bot

Test Your Bot 

Once you have completed all the above steps, your bot will be successfully configured to communicate with users on LINE and is ready to test.

LINE developer console - Navigate to the settings page and you will see, a QR code of your bot.

Mobile LINE app - go to the right most navigation tab with three dots [...] and tap on the QR code icon.

Microsoft Bot Application deploy to Line App using Azure Bot

Live Demo 

Xamarin Developer Interview questions and answers Bot is ready to use in Line app. Xamarin FAQ Bot will be ready with 7000+ more Xamarin QnA’s. Now you can start open your Line App and > click on three dot line > Scan the following QR Code to add Xamarin QA bot as a friend. 

Microsoft Bot Application deploy to Line App using Azure Bot

Summary 

In this article, you have learned how to integrate a bot application into Line App via Azure Microsoft AI. In case your bot is not responding to any of your messages at all, navigate to your bot in Azure portal, and choose Test in Web Chat. If you have any questions/ feedback/ issues, please write in the comment box.

Featured Post

Improving C# Performance by Using AsSpan and Avoiding Substring

During development and everyday use, Substring is often the go-to choice for string manipulation. However, there are cases where Substring c...

MSDEVBUILD - English Channel

MSDEVBUILD - Tamil Channel

Popular Posts