Power Virtual Agents is an easily created powerful bot and no-code graphical interface without the need for data scientists or developers. It eliminates the gap between the subject matter experts and the development teams building the bots and it minimizes the IT effort required to deploy and maintain a custom conversational solution.

This article will demonstrate how easy it is to setup and build a Bot using Power Virtual Agents. The Bot will call a Power Automate Flow to return data from CDS and display it to the user. Power Virtual Agents is part of Microsoft AI and it allows Bots to be created using a Flow like designer without any code

Power Virtual Agents: Build a Microsoft Bot Application

Setup Power Virtual Agents Power Virtual Agents is a software-as-a-service (SaaS). It allows you to easily sign up, create your bot, and embed it into your website and integrate to the different channels with just a few clicks. There’s no infrastructure to maintain or complex systems to deploy.

1. Navigate to Power Virtual Agents and select Start Free and Sign in/ Signup using your work school account.

Power Virtual Agents: Build a Microsoft Bot Application

Create New Bot Application You can create a new bot by selecting the icon on the title bar to open the Bots pane and then selecting New bot, let you select the default environment for now.

Power Virtual Agents: Build a Microsoft Bot Application

Customize your Greeting The Power virtual agent dashboard with sequence steps is the author topic , publish bot and Monitor the performance. You can click on customize your greeting and will see the details below

Power Virtual Agents: Build a Microsoft Bot Application

Select the options “Customize your greeting” and you can see the default user trigger options and message template. Initial trigger phase will be, which user provides the trigger message and based on that bot will reply the message.

Power Virtual Agents: Build a Microsoft Bot Application

Test Your Bot Application You're now ready to test your bot. You will test your bot using the built-in Power virtual agent bot Emulator. Start give the user Input and also refer below image, how to Trigger phase and message sync with user replay


Power Virtual Agents: Build a Microsoft Bot ApplicationPower Virtual Agents: Build a Microsoft Bot Application
Summary 

You have successfully done the setup and built a bot application in the power virtual agent. I hope you have enjoyed learning about the Power virtual agent bot. If you have any questions/feedback/issues, please write them in the comment box.
The Azure AD is the identity provider, responsible for verifying the identity of users and applications and providing security tokens upon successful authentication of those users and applications.in this article I have explained about create Azure AD authentication and integrate into bot application using AuthBot library.



The Bot show very simple dialog with openUrl button and this button launches the web browser for validate user credential and AD will response the message with authentication code, you can copy same code and reply back to the bot, bot will validation and response the welcome message.

You can follow below given steps one by one and you will get to see an interesting demo at end of article.

Azure AD App registration:

I will show the steps given below for the azure application creation, user creation and permission configuration. While implementing bot application, We need Client ID, tenant, return URL, so here I will show how to get all the configuration information from the steps given below.

Step 1: 

Login to Microsoft Azure portal and choose Azure Active Directory from the sidebar.

Step 2:

 If you have not created AZURE Active directory, try to create new AD creation for tenant url or Select or add tenant url from Domain names sections

Building Bot Application with Azure AD Login Authentication using AuthBot

Step 3: 

Select Application Registration and Provide the details given below, name for the application , application type must be Web app/API, enter your application redirect URL and click on Create.

Building Bot Application with Azure AD Login Authentication using AuthBot

Step 4: 

We need to give the permission to access the application from Bot, so grand the permission. Select newly created Application > select Required Permission > Click Grand permission.

Step 5: 

create new user from users and groups sections (optional)

Step 6:

 Create client secret key from Application. Select Application > Select keys > add new / copy client secret key .

Building Bot Application with Azure AD Login Authentication using AuthBot

Step 4: 

You can copy tenant, client ID and Client Secret and you can follow below steps for create and implement AD authentication in Bot

Create New Bot Application:

Let's create a new bot application using Visual Studio 2017. Open Visual Studio > Select File > Create New Project (Ctrl + Shift +N) > Select Bot application.

Building Bot Application with Azure AD Login Authentication using AuthBot


The Bot application template gets created with all the components and all required NuGet references installed in the solutions.

Building Bot Application with Azure AD Login Authentication using AuthBot

Install AuthBot Nuget Package:

The AuthBot provide Azure Active Directory authentication library for implement Azure AD login in Bot .

Right click on Solution, select Manage NuGet Package for Solution > Search “ AuthBot” > select Project and install the package.

Building Bot Application with Azure AD Login Authentication using AuthBot

You can follow given below steps for integrate AD authentication 

Step 1:

 Select Web.config file and add Mode,resourceID,Endpointurl ,Tenant,clientID,clientSecret and redirect url appsettings property and replace Azure AD details as per below
<appSettings>
<add key="BotId" value="YourBotId" />
<add key="MicrosoftAppId" value="" />
<add key="MicrosoftAppPassword" value="" />
<add key="ActiveDirectory.Mode" value="v1" />
<add key="ActiveDirectory.ResourceId" value="https://graph.windows.net/" />
<add key="ActiveDirectory.EndpointUrl" value="https://login.microsoftonline.com" />
<add key="ActiveDirectory.Tenant" value="dxdemos.net" />
<add key="ActiveDirectory.ClientId" value="2d3b5788-05a5-486d-b2a4-2772a4511396" />
<add key="ActiveDirectory.ClientSecret" value="wU3oFBJ1gjWcB8Lo/fMaaCwg7ygg8Y9zBJlUq+0yBN0=" />
<add key="ActiveDirectory.RedirectUrl" value="http://localhost:3979/api/OAuthCallback" />
</appSettings>

Step 2: 

Select Global.asax.cs file and call all the bot app setting property and assign to AuthBot model class, like below
using System.Configuration;
using System.Web.Http;
namespace DevAuthBot
{
public class WebApiApplication : System.Web.HttpApplication
{
protected void Application_Start()
{
GlobalConfiguration.Configure(WebApiConfig.Register);
AuthBot.Models.AuthSettings.Mode = ConfigurationManager.AppSettings["ActiveDirectory.Mode"];
AuthBot.Models.AuthSettings.EndpointUrl = ConfigurationManager.AppSettings["ActiveDirectory.EndpointUrl"];
AuthBot.Models.AuthSettings.Tenant = ConfigurationManager.AppSettings["ActiveDirectory.Tenant"];
AuthBot.Models.AuthSettings.RedirectUrl = ConfigurationManager.AppSettings["ActiveDirectory.RedirectUrl"];
AuthBot.Models.AuthSettings.ClientId = ConfigurationManager.AppSettings["ActiveDirectory.ClientId"];
AuthBot.Models.AuthSettings.ClientSecret = ConfigurationManager.AppSettings["ActiveDirectory.ClientSecret"];
}
}
}

Step 3: 

You can create a new AzureADDialog class to show the default login and logout UI Design dialog. Rightclick on Project, select Add New Item, create a class that is marked with the [Serializable] attribute (so the dialog can be serialized to state), and implement the IDialog interface.
using AuthBot;
using AuthBot.Dialogs;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Bot.Connector;
using System;
using System.Configuration;
using System.Threading;
using System.Threading.Tasks;
namespace DevAuthBot.Dialogs
{
[Serializable]
public class AzureADDialog : IDialog<string>
{

Step 4 :

IDialog interface has only StartAsync() method. StartAsync() is called when the dialog becomes active. The method passes the IDialogContext object, used to manage the conversation.
public async Task StartAsync(IDialogContext context)
{

context.Wait(MessageReceivedAsync);

}

Step 5: 

Create a MessageReceivedAsync method and write the following code for the login and logout default dialog and create a ResumeAfterAuth for after the user login, bot will reply the user name and email id details.
///
/// Login and Logout
///
/// context">
/// item">
///
public virtual async Task MessageReceivedAsync(IDialogContext context, IAwaitable<IMessageActivity> item)
{
var message = await item;
//endpoint v1
if (string.IsNullOrEmpty(await context.GetAccessToken(ConfigurationManager.AppSettings["ActiveDirectory.ResourceId"])))
{
//Navigate to website for Login
await context.Forward(new AzureAuthDialog(ConfigurationManager.AppSettings["ActiveDirectory.ResourceId"]), this.ResumeAfterAuth, message, CancellationToken.None);
else
{
//Logout
await context.Logout();
context.Wait(MessageReceivedAsync);
}
}
///
/// ResumeAfterAuth
///
/// context">
/// result">
///
private async Task ResumeAfterAuth(IDialogContext context, IAwaitable<string> result)
{
//AD resposnse message
var message = await result;
await context.PostAsync(message);
context.Wait(MessageReceivedAsync);
}

After the user enters the first message, our bot will reply and ask to login to the AD. Then, it waits for Authentication code and bot will reply the user details as a response like below.

Building Bot Application with Azure AD Login Authentication using AuthBot

Run Bot Application

The emulator is a desktop application that lets us test and debug our bot on localhost. Now, you can click on "Run the application" in Visual studio and execute in the browser


Building Bot Application with Azure AD Login Authentication using AuthBot
  • Test Application on Bot Emulator
  • You can follow the below steps to test your bot application.
  • Open Bot Emulator.
  • Copy the above localhost url and paste it in emulator e.g. - http://localHost:3979
  • You can append the /api/messages in the above url; e.g. - http://localHost:3979/api/messages.
  • You won't need to specify Microsoft App ID and Microsoft App Password for localhost testing, so click on "Connect".
Building Bot Application with Azure AD Login Authentication using AuthBot

Related Article:

I have explained about Bot framework Installation, deployment and implementation in the below article

Summary

In this article, you learned how to create a Bot Azure AD login authentication and Logout using AuthBot. If you have any questions/feedback/ issues, please write in the comment box.



Microsoft Cognitive services is set of cloud based intelligence APIs for building richer and smarter application development. Cognitive API will use for Search meta data from Photos and video and emotions, sentiment analysis and authenticating speakers via voice verification.

Cognitive Computer Vision API: Intelligent Image Object Detection Bot

The Computer Vision API will help developers to identify the objects with access to advanced algorithms for processing images and returning image meta data information. In this article, you will learn about Computer Vision API and how to implement Compute Vision API into Bot application.


You can follow below steps for implement object detection in Bot Application 
 

Computer Vision API Key Creation

Computer Vision ApI returns information about visual content found in an image. You can follow below steps for create Vision API key.

Navigate to https://azure.microsoft.com/en-us/try/cognitive-services/

Cognitive Computer Vision API: Intelligent Image Object Detection Bot

Click on “Get API Key “or Login with Azure login.

Login with Microsoft Account and Get API key

Cognitive Computer Vision API: Intelligent Image Object Detection Bot

Copy API key and store securely, we will use this API key into our application

Step 2: Create New Bot Application

Let's create a new bot application using Visual Studio 2017. Open Visual Studio > Select File > Create New Project (Ctrl + Shift +N) > Select Bot application.

Cognitive Computer Vision API: Intelligent Image Object Detection Bot

The Bot application template gets created with all the components and all required NuGet references installed in the solutions.

Cognitive Computer Vision API: Intelligent Image Object Detection Bot

In this solution, we are going edit Messagecontroller and add Service class.

Install Microsoft.ProjectOxford.Vision Nuget Package

The Microsoft project oxford vision nuget package will help for access cognitive service so Install “Microsoft.ProjectOxford.Vision” Library from the solution

Cognitive Computer Vision API: Intelligent Image Object Detection Bot

Create Vision Service: Create new helper class to the project called VisionService that wraps around the functionality from the VisionServiceClient from Cognitive Services and only returns what we currently need.

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using System.Web;
using Microsoft.ProjectOxford.Vision;
using Microsoft.ProjectOxford.Vision.Contract;
namespace BotObjectDetection.Service
{
public class VisionService : ICaptionService
{
/// <summary>
/// Microsoft Computer Vision API key.
/// </summary>
private static readonly string ApiKey = "<API Key>";
/// <summary>
/// The set of visual features we want from the Vision API.
/// </summary>

private static readonly VisualFeature[] VisualFeatures = { VisualFeature.Description };

public async Task<string> GetCaptionAsync(string url)
{
var client = new VisionServiceClient(ApiKey);
var result = await client.AnalyzeImageAsync(url, VisualFeatures);
return ProcessAnalysisResult(result);
}
public async Task<string> GetCaptionAsync(Stream stream)
{
var client = new VisionServiceClient(ApiKey);
var result = await client.AnalyzeImageAsync(stream, VisualFeatures);
return ProcessAnalysisResult(result);
}
/// <summary>
/// Processes the analysis result.
/// </summary>
/// <param name="result">The result.</param>
/// <returns>The caption if found, error message otherwise.</returns>

private static string ProcessAnalysisResult(AnalysisResult result)
{
string message = result?.Description?.Captions.FirstOrDefault()?.Text;
return string.IsNullOrEmpty(message) ?
"Couldn't find a caption for this one" :
"I think it's " + message;
}
}
}


In the above helper class, replace vision API key and call the Analyze image client method for identify image meta data

Messages Controller

MessagesController is created by default and it is the main entry point of the application. it will call our helper service class which will handle the interaction with the Microsoft APIs. You can update “Post” method like below

using System;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using System.Web.Http;
using BotObjectDetection.Service;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Bot.Connector;

namespace BotObjectDetection
{
[BotAuthentication]
public class MessagesController : ApiController
{
private readonly ICaptionService captionService = new VisionService();
/// <summary>
/// POST: api/Messages
/// Receive a message from a user and reply to it
/// </summary>

public async Task<HttpResponseMessage> Post([FromBody]Activity activity)
{
if (activity.Type == ActivityTypes.Message)
{
//await Conversation.SendAsync(activity, () => new Dialogs.RootDialog());
var connector = new ConnectorClient(new Uri(activity.ServiceUrl));
string message;
try
{
message = await this.GetCaptionAsync(activity, connector);
}
catch (Exception)
{
message = "I am object Detection Bot , You can Upload or share Image Url ";
}
Activity reply = activity.CreateReply(message);
await connector.Conversations.ReplyToActivityAsync(reply);
}
else
{
HandleSystemMessage(activity);
}
var response = Request.CreateResponse(HttpStatusCode.OK);
return response;
}

private Activity HandleSystemMessage(Activity message)
{
if (message.Type == ActivityTypes.DeleteUserData)
{
// Implement user deletion here
// If we handle user deletion, return a real message
}
else if (message.Type == ActivityTypes.ConversationUpdate)
{
// Handle conversation state changes, like members being added and removed
// Use Activity.MembersAdded and Activity.MembersRemoved and Activity.Action for info
// Not available in all channels
}
else if (message.Type == ActivityTypes.ContactRelationUpdate)
{
// Handle add/remove from contact lists
// Activity.From + Activity.Action represent what happened
}
else if (message.Type == ActivityTypes.Typing)
{
// Handle knowing tha the user is typing
}
else if (message.Type == ActivityTypes.Ping)
{
}
return null;
}
private async Task<string> GetCaptionAsync(Activity activity, ConnectorClient connector)
{
var imageAttachment = activity.Attachments?.FirstOrDefault(a => a.ContentType.Contains("image"));
if (imageAttachment != null)
{
using (var stream = await GetImageStream(connector, imageAttachment))
{
return await this.captionService.GetCaptionAsync(stream);
}
}
string url;
if (TryParseAnchorTag(activity.Text, out url))
{
return await this.captionService.GetCaptionAsync(url);
}
if (Uri.IsWellFormedUriString(activity.Text, UriKind.Absolute))
{
return await this.captionService.GetCaptionAsync(activity.Text);
}
// If we reach here then the activity is neither an image attachment nor an image URL.
throw new ArgumentException("The activity doesn't contain a valid image attachment or an image URL.");
}

private static async Task<Stream> GetImageStream(ConnectorClient connector, Attachment imageAttachment)
{
using (var httpClient = new HttpClient())
{
var uri = new Uri(imageAttachment.ContentUrl);
return await httpClient.GetStreamAsync(uri);
}
}

private static bool TryParseAnchorTag(string text, out string url)
{
var regex = new Regex("^<a href=\"(?<href>[^\"]*)\">[^<]*</a>$", RegexOptions.IgnoreCase);
url = regex.Matches(text).OfType<Match>().Select(m => m.Groups["href"].Value).FirstOrDefault();
return url != null;
}
}
}


Run Bot Application 

The emulator is a desktop application that lets us test and debug our bot on localhost. Now, you can click on "Run the application" in Visual studio and execute in the browser

Cognitive Computer Vision API: Intelligent Image Object Detection Bot

Test Application on Bot Emulator


You can follow the below steps to test your bot application.

Open Bot Emulator.

Copy the above localhost url and paste it in emulator e.g. - http://localHost:3979

You can append the /api/messages in the above url; e.g. - http://localHost:3979/api/messages.

You won't need to specify Microsoft App ID and Microsoft App Password for localhost testing, so click on "Connect".

Cognitive Computer Vision API: Intelligent Image Object Detection Bot

Related Article: I have explained about Bot framework Installation, deployment and implementation in the below article

Summary


In this article, you learned how to create an Intelligent Image Object Detection Bot using Microsoft Cognitive Computer Vision API. If you have any questions/feedback/ issues, please write in the comment box.
The Bot Application runs in  an application, like Skype, web chat, Facebook, Message, etc. Users can interact with bots by sending them messages, commands, and inline requests. You control your bots using HTTPS requests to the bot API.

The better visibility, the higher will be the chance of customers making a purchase. Unlike other bots which are available on business web pages which provide visibility only to the customers visiting their site, a Facebook bot is very easy to find. It is visible to around 1.3 billion users of Facebook. A Facebook bot interacts very well with customers. It can receive and send messages, images and provide clickable buttons like Buy Now. In this article, I am going to show how we can connect Facebook Messenger's channel and integrate Bot Application to the Messenger app.

In this Article, I am going to show how we can connect Facebook messengers’ channel and integrate Bot Application to the messengers App.


Create FAQ Bot Application

You can refer my previews article to create and build a Xamarin FAQ Bot using the Azure Bot Service and deploy it into Azure. I am not using any coding for developing the Bot Application, you can follow the steps provided in the article to create and deploy FAQ Bot.


Setup Facebook Page

We can implement Bot Application to the Facebook page. You can create Facebook page or select existing page and navigate to “About page” for fining and copy the Page ID.


Login to Facebook APP

Create a new Facebook App (https://bit.ly/1D0BHpg) on the Page and generate an App ID , App Secret and Page Access Token for integrating Bot to the page messenger. You can click on “Skip and Create APP Id” from following screen.




Create New App ID



Provide display Name and Contact Email for integrating Bot application and Click on Create.


After Create button, it will navigate to the App dashboard screen. The side navigation menu having Settings > Basic and copy the APPID and APP Secret. Provide the Privacy URL, Terms of Service URL, App icon and Select Category.




I have shown the following screen to Select Setting > Advanced and Set the "Allow API Access to App Settings slider to "Yes" and click on Save Changes.

Setup Message

Select Dashboard and Click on Setup button from messenger group


Create Page Access Token

Select Setting from the messenger side navigation menu to generate token, you can select the Page and generate access token and copy the page access token.

Setup Webhooks

Click Set up Webhooks to forward messaging events from Facebook Messenger to the bot.



Provide following callback URL and Verify token to the webhooks setup page and select the message, message_postbacks, messaging_optins and message_deliveries subscription fields. The following Steps 2, will show how to generate Callback URL and verify token from Azure portal.


You can click on “Verify and Save “and select the Facebook page to Subscribe the webhook to the Facebook page.

Connect Facebook Channel

Step 1: Login to Azure portal > Select the “All Resources” > Select Channels > Select Facebook Messengers, let we start configure “Facebook Messengers “Channel and follow below steps, the end of this article you can able to deploy Bot into the Facebook messenger



Step 2:The Azure Facebook configuration channel will generate following Callback URL and verify token, You can copy those information and Past to the Facebook webhook setup screen. (Return to the Facebook messenger setup screen).






Step 3: You can paste the Facebook App ID, Facebook App Secret, Page ID,and Page Access Token values copied from Facebook Messenger previously. You can use the same bot on multiple facebook pages by adding additional page ids and access tokens.


Submit to Facebook Review

Select App preview and Click and Submit for review after submit will take some time for the facebook team testing to the messenger bot and you can mark your app live available to the public , then you can test messenger from the Facebook page .


Facebook Team reviews and Testing

You can verify Facebook team testing progress, navigate to your Facebook page and Click on Inbox and verify, if anything problem to the messenger bot, the Team will update the bug list from App review screen.

Xamarin FAQ Messengers testing

You can select your Facebook page and test your bot application. I have trained 7000+ more questions to the Facebook messenger bot from Xamarin Q & A Page, If you want to look the xamarin FAQ demo. Navigate to Xamarin Q & A Facebook page and ask your xamairn related question.





Summary

In this article, you have learned how to integrate bot application right into your Facebook Page via Azure Microsoft AI. If you have any questions/feedback/issues, please write in the comment box.


Microsoft Cognitive Services offers awesome APIs and services for developers to create more intelligent applications. You can add interesting features, like people's emotions and video detection, face, speech, and vision recognition and speech and language understanding into your application.
The Emotion API takes a facial expression in an image stream as an input and returns confidence levels across a set of emotions for each face in the image like anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise, in a facial expression
Microsoft AI: Mobile Emotion Recognition Application

Where to implement 

There is no single unifying definition or theory of emotion. Nevertheless, several characteristics of an emotional reaction are frequently mentioned. Nowadays feedback is more important for product promotion and review the article, store, etc. manual entry of emotion maybe wrong so we can automate the feedback and review.

Microsoft AI: Mobile Emotion Recognition Application

In this blog, we can learn about how to implement Emotion Recognition in Xamarin.Forms with Microsoft Cognitive Services.

Getting Started with the Emotion API 

The Emotion API can detect anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise, in a facial expression. As well as returning an emotion result for a facial expression, the Emotion API also returns a bounding box for detected faces using the Face API. If a user has already called the Face API, they can submit the face rectangle as an optional input. Note that an API key must be obtained to use the Emotion API. You can follow below steps for obtained at free API key on Microsoft.com

Step 1: Click here for Generate Subscriber Key and Click on Create


Microsoft AI: Mobile Emotion Recognition Application
Step 2: You can agree the terms and Microsoft privacy statement > select the country and click on Next

Microsoft AI: Mobile Emotion Recognition Application

Step 3: Login with Microsoft /LinkedIn /Facebook /Github for create your API

Microsoft AI: Mobile Emotion Recognition Application

Step 4: You login and subscribe you should see the keys and the below information on available your subscriptions 

Microsoft AI: Mobile Emotion Recognition Application

We will use emotion API key in the following Xamarin application. before that we can test the API key

Test on Your Emotion Key

You can navigate emotion API online console application and provide the subscription key and image url . If your provide valid key and input the image, you will get it JSon data as output or relevant error

Microsoft AI: Mobile Emotion Recognition Application

Create New Xamarin Forms Application 

You can refer my previous article for setup and create new Xamarin forms application.

If you already installed VS on your machine, Open Visual Studio > Create New Xamarin Forms application like below

Microsoft AI: Mobile Emotion Recognition Application

Solution will create with all the platform and PCL project

Microsoft AI: Mobile Emotion Recognition Application

In this solution, we are going to create demo application for Article review automate based on the emotion.

Camera in Xamarin. Forms 

We can use Media.Plugin Nuget package for Capture image or select image gallery from all the mobile platform (iOS,Android,Windows)

Right Click on Solution > Type “xam.plugin” in the search box > Select all the project (PCL, iOS, Android and UWP) > Click on Install

Microsoft AI: Mobile Emotion Recognition Application

Emotion Nuget Package

 This client library is a thin C# client wrapper for the Microsoft Emotion API.The easiest way to use this client library is to get microsoft.projectoxford.emotion package from nuget .

Right Click on Solution > Type “Microsoft Emotion” in the search box > Select all the project (PCL, iOS, Android and UWP) > Click on Install

Microsoft AI: Mobile Emotion Recognition Application

UI Design 

After successfully install above two nuget package. Let start UI design from PCL project In Dotnet Standard Project > Open MainPage.Xaml page and design following xaml code


<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
xmlns:local="clr-namespace:DevEnvExeEmotion"
x:Class="DevEnvExeEmotion.MainPage">
<StackLayout Margin="20">
<StackLayout Orientation="Horizontal" Padding="30" HorizontalOptions="CenterAndExpand" >
<StackLayout x:Name="emotion">
<Label Text="Reader is :" FontSize="Large"/>
<Label x:Name="emotionResultLabel" FontSize="Large" />
</StackLayout>
<Button Text="Share Your FeedBack" Clicked="OnFeedBackClicked"></Button>
</StackLayout>
<Image x:Name="image" Source="article.png" />
<ActivityIndicator x:Name="activityIndicator" />
</StackLayout>
</ContentPage> 


In mainPage.xaml.cs file add your logic.


public partial class MainPage : ContentPage
{
EmotionServiceClient emotionClient;
MediaFile photo;
public MainPage()
{
InitializeComponent();
emotion.IsVisible = false;
emotionClient = new EmotionServiceClient("7f495be5faf643adbeead444d5b79a13");
}
For capture image, write the following code
public async Task CaptureImage()
{
await CrossMedia.Current.Initialize();
emotion.IsVisible = false;

// Take photo
if (CrossMedia.Current.IsCameraAvailable || CrossMedia.Current.IsTakePhotoSupported)
{
photo = await CrossMedia.Current.TakePhotoAsync(new StoreCameraMediaOptions
{
SaveToAlbum = false
});
}
else
{
await DisplayAlert("No Camera", "Camera unavailable.", "OK");
}
}
For Recognize emotion, write the following code
public async Task Recognizeemotion()
{
try
{
if (photo != null)
{
using (var photoStream = photo.GetStream())
{
Emotion[] emotionResult = await emotionClient.RecognizeAsync(photoStream);
if (emotionResult.Any())
{
// Emotions detected are happiness, sadness, surprise, anger, fear, contempt, disgust, or neutral.
emotionResultLabel.Text = emotionResult.FirstOrDefault().Scores.ToRankedList().FirstOrDefault().Key;
emotion.IsVisible = true;
}
photo.Dispose();
}
}
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
}

While click on Feedback button, call the above two methods

async void OnFeedBackClicked(object sender, EventArgs e)
{
//capture image
await CaptureImage();
((Button)sender).IsEnabled = false;
// Recognize emotion
await Recognizeemotion();
((Button)sender).IsEnabled = true;
}

Run the Application

Select the iOS /Android /Windows and install into the device and click on ‘Share your Feedback ‘button
Microsoft AI: Mobile Emotion Recognition Application

The device should be camera so after click on the button, capture your face for Recognize emotion.



Microsoft AI: Mobile Emotion Recognition Application

Summary


 In this article, you learned about how to generate emotion API key and implement Emotion Recognition in Xamarin.Forms with Microsoft Cognitive Services.If you have any questions/ feedback/ issues, please write in the comment box. 

Featured Post

Power Virtual Agents: customize the bot conversation messages

Power Virtual Agents are an easily created powerful bot and no-code graphical interface without the need for data scientists or developers. ...

Subscribe to the FREE Weekly Newsletter to receive all the latest Microsoft Azure, AI, Xamarin

Popular Posts