QnA Maker – Documents to a FAQs App

A common challenge for most informational FAQs (Frequently Asked Questions) scenarios is to separate out the content management from the FAQs design and development, as content owners are usually domain experts who may not be technical. QnAMaker addresses this by enabling a QnA management experience.

In Microsoft words – Azure QnA Maker allows you to edit, remove, or add QnA pairs with an easy-to-use interface, then publish your knowledge base as an API endpoint for a bot service. It’s simple to test and train the bot using a familiar chat interface, and the active learning feature automatically learns questions variations from users over time and adds them to your knowledge base. Use the QnA Maker endpoint to seamlessly integrate with other APIs like Language Understanding service and Speech APIs to interpret and answer user questions in different ways. More details are available here.

To get my hands dirty, I started developing a minimal app. Basically, a Hello World with QaA Maker

The high-level scope is –

  1. Firstly, I have an “Azure Purchase FAQ.pdf” file which contains the questions and answers related to Azure purchasing (it’s a cut short version for obvious reasons).
  2. Secondly, I uploaded the above file to https://www.qnamaker.ai/ and terminology wise, it’s the knowledge base creation.
  3. Lastly, developed a quick console application using C#. On start, this app hits the ‘qnamaker’ endpoint to search a question and gets the answer for it.

Honestly, writing this blog took me more time than developing an end to end working solution

The solution is uploaded on the GitHub with project namely, “AzurePurchaseFAQ“. Feel free to copy/clone and try your own hands also. To begin, create the QnA Maker service and deploy with free/defaults. The settings for Sample HTTP request would look like –

POST /knowledgebases/ee0f5e4a-ac42-4f91-9b47-8161b6c5a409/generateAnswer

Host: https://azurepurchasefaq.azurewebsites.net/qnamaker

Authorization: EndpointKey 4836aac3-9fcf-45ca-9295-d256a50216ec

Content-Type: application/json


On running the app, the console window should look like this –

Happy coding, and have fun 🙂

Big Data Concepts – In 5 Minutes

What is Big Data –

If you are looking for standard definition, then refer to obvious source i.e. wiki

As per wiki, the term has been in use since the 1990s, with some giving credit to “John Mashey” for coining or at least making it popular. Big data usually includes data sets with sizes beyond the ability of commonly used software tools to capture, curate, manage, and process data within a tolerable elapsed time. More details are anyways at wiki.

The definition I prefer is, “When data is too big for OLTP then it’s Big Data“. Other definitions –

  • When data is in Peta Bytes.
  • 3 Vs (Volume, Velocity and Variety) or 4Vs (Volume, Velocity, Variety, and Veracity)

What Scenario produces it –

Data getting produced from web/internet, social networking/media, phone/mobile tower and many more as mentioned in the diagram below.

Point to be notes is, the notion of big data is not NEW. We always had it, what we haven’t done is to STORE IT and ANALYSE IT. This is now possible because of many factor/enablers.

What Enables it –

If you compare today with a day decades ago. You will observe the entry barriers got reduced significantly and democratization of concepts and its enablers happened. For example, nowadays buying compute/storage resources is relatively cheap than it was previously. Also, the technologies/solutions required to make sense out of big data are more accessible, thanks to open source initiatives and its serious players in the market. Hence, today we have more and more Producers and Consumers of data who are interested in it and its analysis.

I’m trying to list few enablers, but true list would be far greater than this. However, it should give you initial food for thoughts.

What It Enables –

  • Analysis – Sentiments, Clickstream and Forensic etc. Analysis.
  • Patterns – Buying, Search and Investment.
  • Machine Learning
  • Research – Physics and Healthcare
  • Prediction and Prevention Maintenance.
  • And many more…Just Bing/Google it

Map Reduce, I heard somewhere about it what’s that –

Developed and perfected inside the google then published to public. It’s 2 pass process – 1) Map and 2) Reduce. More details

Let’s understand it quickly via picture. As, “a picture is worth a thousand words”

Although picture is self-explanatory, but I will add the explanation, if required and requested