A Guide to Develop AI Powered Chatbots

World is evolving faster, we can see AI everywhere. One of the greatest examples is launch of Alexa & Google home. They are creating buzz around the people.

Let’s get into world of AI, we are going to develop our own bot. There are many types of bot, popular once are “Retrieval-Based Bots” and “Generative based Bots”.

Retrieval Based Chatbot: Retrieval models have a pre-defined responses, which is unlike generative
models that can generate responses they’ve never seen before. Rule-Based bot is one of the example. The rules on which it works can vary from simple to very complex. In rule-based, intent and entity of the question is found and, on that basis, user gets the answer.

Generative Based Chatbot: This is a type of bot which requires lots of training data as it is mainly developed with deep learning models. We can develop a generative based chatbot using simple RNN(LSTM) model.

Generally, most of the problems can be solved using a rule-based approach.

Today, we are going to develop a Rule-based bot. We are going to use Microsoft Azure Services to build this bot. Microsoft Cognitive services provide lots of API’s like Vision API, Knowledge API (QnA Maker), Language API (LUIS), Speech API & Search API (Azure Search).

Vision API: Image-processing algorithms to smartly identify, caption and moderate your pictures.

Knowledge API: Map complex information and data to solve tasks such as intelligent recommendations and semantic search.

LUIS: Allow your apps to process natural language with pre-built scripts, evaluate sentiment and learn how to recognize what users want.

Speech API: Convert spoken audio into text, use voice for verification or add speech recognition to your app.

Search API: Add Bing Search APIs to your apps and harness the ability to comb billions of web pages, images, videos and news with a single API call.

INTRODUCTION:

We are going to use below components for our chatbot:

  1. Bot Framework: The Microsoft Bot Framework helps us to build, connect, deploy, and manage intelligent bots to naturally interact with your users on a website, app, Cortana, Microsoft Teams, Skype, Slack, Facebook Messenger, and more. Get started quickly with a complete bot building environment, all while only paying for what you use. We are going to write our code in dot net.
  2. QnA Maker: QnA Maker is an easy-to-use web-based service to train AI to respond to user’s questions in a more natural, conversational way. We can build, train and publish a simple question and answer bot based on FAQ URLs, structured documents, product manuals or editorial content in minutes.
  3. Microsoft Azure Services: Microsoft Azure is a cloud computing service created by Microsoft for building, testing, deploying, and managing applications and services through a global network of Microsoft-managed data centers.Note: Azure account starting is free, plus you get a ₹13,300 credit to spend in the first 30 days. You will not be charged — even if you start using services — until you choose to upgrade.PLAN FOR BOT DEVELOPMENT:
    We are going to follow below step for the chatbot creation:1. Go to https://www.qnamaker.ai/ and sign in:
    Creating our Knowledge Base using QnA Maker & Microsoft Azure account creation:
    Now click on “Create a knowledge base “:This part consists of 5 parts are as follows :
    Step 1: Create a QnA service in Microsoft Azure. When you will click on “Create a QnA Service”, it will take you to Microsoft Azure page and you have to create a new Knowledge based service

    Now provide all the details to create QnA Service:

    Once service is created you can Connect your QnA service to your KB from step 2 in QnA maker page:

    QnA maker Step 2 for knowledge base creation

    Now provide name of your Knowledge Base in step 3:

     

    Providing Knowledge base name

    Now provide the Knowledge base in step 4. It can Extract question-and-answer pairs from an online FAQ, product manuals, or other files. Supported formats are .tsv, .pdf, .doc, .docx, .xlsx, containing questions and answers in sequence. You can also add the Question & Answer manually. I have created my own Question and Answer and created a Mall assistant bot , We can use this URL for demo  https://docs.microsoft.com/en-us/azure/bot-service/bot-service-resources-bot-framework-faq?view=azure-bot-service-3.0 .

     

       Now click on “Create your KB” to create your knowledge base at step 5

     

    Here my knowledge based is created:

    Now you can test your knowledge base by clicking on TEST button:

    You can do lots of thing in knowledge base, like adding more questions, modifying your questions & adding similar kind of question for an answer etc. It makes the bot more efficient. You can refer to QnA maker documentation to learn more about it.

    Now click on publish tab and publish your knowledge base and you get below details:

    2. Creating Web bot service in Microsoft Azure:

     We are done with QnA maker setup now. Let’s go to Microsoft Azure account and click on create
    resource -> AI + Machine Learning -> Web App :

    Now you can access your Web App bot by clicking on Web App Bot i.e Assistant bot here :

    3. Setup of Visual studio:

     Go to the location https://visualstudio.microsoft.com/vs/older-downloads/ and download Visual   Studio 2015

     4. Setup Bot Framework:

      Go to your App web bot -> click on build -> click on Download zip file

    Once it is downloaded, unzip it and open Microsoft.Bot.Sample.QnABot

    Click on Microsoft.Bot.Sample.QnABot

    Here is the visual studio:

    Message Controller: A Message Controller helps the user and bot to get connected. When the user responds first the message goes to message controller then it will help to connect with dialogs.

    Dialog: A dialog specify the action to be taken by bot on receiving a message from the user.

    Now go to dialog-> BasicQnAMakerDialog.cs

    Microsoft have done some changes in the services and syntax of the code. You can copy the below codes and paste it to the BasicQnAMakerDialog.cs :

    5. Integration of Bot Framework with Azure services and QnA Maker:

    Now go to Web.config and add MicrosoftAppId, MicrosoftAppPassword , AzureWebJobsStorage,   QnAEndpointHostName , QnAAuthKey & QnAKnowledgebaseId :

    MicrosoftAppId, MicrosoftAppPassword & AzureWebJobsStorage can be found at Web App Bot ->   Application Setting as shown below:

    MicrosoftAppId, MicrosoftAppPassword & AzureWebJobsStorage

    QnAEndpointHostName , QnAAuthKey , QnAKnowledgebaseId can be found in end of step 1 :

    Published Key

    Now provide QnAEndpointHostName , QnAAuthKey , QnAKnowledgebaseId at Web App Services ->  Application Setting

    QnAEndpointHostName , QnAAuthKey , QnAKnowledgebaseId

    Now go to the Visual studio and click on rebuild

    Once rebuild is done go to Microsoft Azure -> App Service

    Click on the Get publish profile, your password will get downloaded:

    Now go to visual studio and click on publish as shown below:

                                                           Publishing the project

    It will ask for password, open Get publish profile and get the password as shown below:

    Provide publish profile Password

    Getting password for publishing project

    Once the bot get published successfully, go to Microsoft Azure -> Web App Bot -> Channel ,    shown  below:

    Click on skype and login add it to your skype id:

    6. Chat with the Bot :

     

    Github: https://github.com/ranjan-sumit/Chatbot-DotNet

    Happy Learning !!

    Name : Sumit Ranjan

    Designation : Data Scientist

Data Augmentation For Deep Learning Algorithms

Plentiful high-quality data is the key to great deep learning models. But good data doesn’t come easy, and that scarcity can impede the development of a good model. It’s relatively easy for a model to identify things from an image when everything is ‘Just Right’ —the correct illumination and zoom level, the right perspective etc. But when the images are not so ideal, the model struggles to give a reasonable prediction. So typically, one would want to train a network with images which are ‘not so ideal’ to help it predict better. But how do you get such data? Well, you basically fake it by taking a regular image and using data augmentation.

DOCKER FOR DATA SCIENCE

For the past 3 years, I have heard a lot of buzz about docker containers. I wanted to figure out about this technology & how it could help for a productive developer or data scientist.

I tried to convey my findings through this blog so you don’t need to parse all the information out there. Let’s get started.

Docker?
Docker is an open-source project based on Linux containers. It uses Linux Kernel features like namespaces and control groups to create containers on top of an operating system. Docker is a tool designed to make easier to create, deploy and run applications by using containers.

We can think docker as lightweight virtual machines that contain everything you need to run an application. Even biggies like Google, Amazon, VMware have built services to support it. That’s all you need to know about docker for now.

ChatBoTs to Cognitive Conversational Personalized Assistants (CCPA’s) – A necessary metamorphosis

ChatBots have been evolving for almost 3 years now. However, by 2018, ChatBots stand at a cusp, with the technology and usage, ready to take off in an exponential way; powered by AI, NLP, and Analytics. ChatBots are intended to provide end users with a Cognitive, Conversational and Customized Interface to get their job done.

Isolating Toxic Comments to prevent Cyber Bullying

Online communities are susceptible to Personal Aggression, Harassment, and Cyberbullying. This is expressed in the usage of Toxic Language, Profanity or Abusive Statements. Toxicity is the use of threats, obscenity, insults, and identity based hateful language.

Computer Vision & NLP

Learn How to Classify Documents Using Computer Vision and NLP

Many companies, especially those in BFSI and Legal sectors, deal with a large volume of handwritten and scanned documents. It is difficult to easily use the granular information in these documents to perform an analysis or even browse through the documents in a convenient manner. A simple classification of the documents into meaningful bins or folders would make it a lot easier to leverage the information within the documents.

Transmorphosizing Banking Through Artificial Intelligence

Banking is re-inventing itself as it always has

“Help us build the kind of bank you want to use” – That’s what Monzo says to its customers. Monzo is a UK based banking app with no usage fees, no branches, no mortgages and zero charges for spending abroad. While few of the banks in the UK might have a similar offer, but where Monzo has gained grounds is how it emotionally connects to its users. It auto-budgets your expenses, highlights where you are spending more, helps in finding the best deal e.g. it puts the money in the best savings accounts. They have formed a partnership with Google and Amazon to ensure real-time banking. For generations of users who spend a considerable amount of their time on their smartphones, it’s more than a conventional bank. Not surprising, they are termed as the new “wonder kid” in the banking industry and are aiming to have a billion-user base in next 5 years (ref: www.thegaurdian.com).

New Product Forecasting using Deep Learning – A unique way

Background

Forecasting demand for new product launches has been a major challenge for industries and cost of error has been high. Under predict demand and you lose on potential sales, overpredict them and there is excess inventory to take care of. Multiple research suggests that new product contributes to one-third of the organization sales across the various industry. Industries like Apparel Retailer or Gaming thrive on new launches and innovation, and this number can easily inflate to as high as 70%. Hence accuracy of demand forecasts has been a top priority for marketers and inventory planning teams.

Product Life Cycle Management in Apparel Industry

Product Life Cycle Estimation

“Watch the product life cycle; but more important, watch the market life cycle” – Philip Kotler

A. Abstract

The product life cycle describes the period over which an item is developed, brought to market and eventually removed from the market.

This paper describes a simple method to estimate Life Cycle stages – Growth, Maturity, and Decline (as seen in the traditional definitions) of products that have historical data of at least one complete life cycle.

Here, two different calculations have been done which helps the business to identify the number of weeks after which a product moves to a different stage and apply the PLC for improving demand forecasting.

A log-growth model is fit using Cumulative Sell through and Product Age which helps to identify the various stages of the product. A Log-Linear model is fit to determine the rate of change of product sales due to a shift in its stage, cet. par.

The life span of a product and how fast it goes through the entire cycle depends on market demand and how marketing instruments are used and vary for different products. Products of fashion, by definition, have a shorter life cycle, and they thus have a much shorter time in which to reap their reward.