Overview:

This document describes an approach and steps that will help organize and plan your bot deployment. Building a quality bot experience requires a few basic steps and a structured approach.

• Our approach starts with defining initial goals and scope for the first iteration. Once the initial version is tested and validated, it can be rolled out to live customers. The iterative improvement process begins after that. Monitoring usage and performance followed by adaptation cycles is the key to improving performance. As your bot improves over time, you will further define the capabilities and expand the scope of knowledge and tasks supported.
• This document describes the process and approach. Please see the Koopid Provider Portal User Guide for specific details on leveraging the Koopid tools to implement your bot
• Iterate, build on successes, offer good fallback options, extend based on real data and feedback

Initial Scope

• The first step creating your bot experience is to define the types of tasks and questions the bot will handle. This may be done through your own experience and knowledge of your customers but will be greatly improved with access to existing data. The goal is to determine the variety and volume of interaction types your customers are already performing or inquiring about. If your goal is to have the most impact, look for the highest volume. If your goal is to cover the most items, study and categorize the array of transaction types.
• Common sources of data include:
  • Chat transcripts and call recordings
  • Web traffic analysis
  • Existing FAQ documents or pages
  • IVR reports
• Tasks: Identify any tasks you want the bot to perform. These are processes that can be expressed in a workflow or through a conversational dialog. Tasks may be static multi-step processes or dynamically driven by identity, topic, and other sources of integrated data. This might include: ticket status, order confirmation, etc.
• Intents: Intents identify items that may not have a current task associated with them but should be detected for classification and potential routing to live assistance. These can be linked to workflows that handle routing only and provide the path for future expansion as more automation is introduced into the system.
• Q&A: Questions & Answers, or Frequently Asked Questions (FAQ) are static responses to common questions. These enable self-service automation for simple questions that are asked repeatedly. Many organizations already have documents and data sources that contain these types of Q&A details.

Training

• Once the structure is in place for what your bot will do, training phrases must be defined and added. For each of your workflows (intents, tasks) and Q&A answers, you will add specific training phrases, sentences, and terms. See the Koopid Portal documentation for the mechanics of training.
• First start by defining 10 – 20 phrases for each task, intent, and Q&A item. This initial set is designed to provide a functional release that will be adapted or taught over time.
• Be sure to include common terms, product names, abbreviations, and variations of terminology your customers use to discuss the task, intent, or Q&A. For example, if you have a workflow to display order status you may use training phrases like the following:
• For example, if you have a workflow to display order status you may use training phrases like the following:
  • I need to check on my order
  • What is the current status of my order
  • Has my order shipped
  • Order status
  • Check my order
  • Is my order ok
• Note, these sentences and phrases will be used to train the AI engine. While more data is better when training AI, the engine is flexible and will create a model that will match related phrases in addition to the actual phrase.

*Q&A Content: For Q&A items you will also need to enter the answer for each series of phrases. Ensure answers to Q&A content are not too long. Where appropriate provide a link to further information. Remember the customer interface and limitations. Scrolling content may not be preferred, text combined with a link to more information may be the most effective.

• Word Choice Specifics

  • Misspellings may be handled in a few different ways:
  • For common misspellings your customers use a lot, add phrases with those into your model
  • Many input clients leverage spellcheck and will correct the misspellings before they reach the system.
  • Sometimes ignoring the misspelling may be ok. Classification will still work well based on the strength of the other surrounding words.
  • Enter your training phrases into the Koopid XDesigner for each workflow and Q&A item.

FallBack WorkFlow

• Fallback is an important concept to handle customer input when the scores are low or a matching intent can’t be found. Through the Koopid XDesigner, you can define the Fallback Workflow. This workflow is entered when the user entry does not match anything in the bot model. This workflow should gracefully ask the user for clarification and move the user to the desired action or routing to a human agent.

Testing

• Once the initial bot models are defined, testing should be performed before wider rollout. This testing should involve a few types of input. If the results of a test are incorrect, then the training data above should be extended or modified to achieve the desired performance.
  • Positive tests: These are phrases that should be matched by one of your defined classifications
  • Negative tests: These are tests that should not match any of your defined classifications
  • Out of vocabulary tests: These are tests that should not match and are very different from the training data. They should go to fallback.

Overlap

One common problem may be overlap. This is a condition when you have two or more sets of training data that have entries too close to each other, or very similar. In this case the model may produce an undesired classification due to this closeness.

• An example might be two phrases like:
  • order status
  • place order
• In this case there may not be enough difference between the two.
• This problem can be corrected by adding more phrases and sentences to each training set to create a larger vocabulary for each set and distinguish between the items. In the case above adding more phrases that us “place” for the new order classification and more phrases with “status” for the order status classification will help strengthen the differences in the models.
• Alternatives: When dealing with potential overlap areas it may be useful to present validation to users when they enter areas that are easily confused. This may take the form of confirmation questions “Are you sure you want to cancel your order?”, or choice buttons to clarify a specific variant of a task “Did you want to search for a new or used car?”

Learning

Once you have deployed or tested your bot improvements will be required. Learning is an iterative process. The more real-world usage and analysis the better your bot will become.

• Analysis: The first step to learning is to analyze the data collected through the Koopid Portal. The current usage data may be extracted and analyzed for performance. The extracted data will contain the actual phrases, the classification detected, the score, and sentiment. The extracted data should be examined for three different cases:
  • Look at items for each classification to ensure the correct classification occurred. Even if the correct classification occurred, it may be useful to add any new phrases to the training set. This will reinforce the model with more data making it more robust.
  • Look at items for each classification that were incorrect. These should be added to the correct classification for retraining. If they are not covered by a classification then they can be added to a fallback classification to help pull them away from your defined classifications.
  • Look at items classified as fallback. These are items that did not match. If any of these items belong in a classification, add them to the model.
• Adaptation: Based on the above analysis, add or adjust the classification models. You may need to add some new intents, tasks, or Q&A content to accommodate things you learned your customers are asking about. This adaptation process will occur automatically when you save the changes to your models. They will be available immediately to your customers.
  • Export Data:
  • From the learn-extract menu in XDesigner download the json file for a given date range. Search the result for high and low scoring items.

 

• Reinforced Learning: Teach each classification to learn new phrases and differentiate to reduce error cases.

 

 

Entities

• Entities enable your bot to extract pieces of information from the input phrase. The result becomes usable in a variable for leverage in your workflows. Built-in entities are available for common items like phone, email, names, etc. In addition, you can define custom entities to use for detection of product names, or other custom items.
  

Workflows and Dialogs

• Customer interaction is more than classification of a single input. A few style suggestions for dialogs and workflow interactions will help you craft effective, stylish interfaces
  • Anticipate next steps and present options where possible
  • Clearly prompt the user when input is needed with cues on type of input
    • provide immediate feedback if input does not conform to what is expected
    • provide a helpful hint on how to correct the input
  • Only where appropriate – enable the user to provide feedback on the last flow step
  • Be economical with prompts, but avoid being cryptic
  • If a response to user input will take time, provide indication to user that system is working to deliver a response
  • Control the input box to only be offered where input is possible.
  • Use deterministic choice instead of free-form input where possible.