IBM WATSON - INTENT REFINEMENT

MBFS

UX DESIGN

UI DESIGN

ABSTRACT

/Finished Composition

THE PROBLEM

MBFS utilizes an artificial intelligence chatbot that had issues with understanding what users typed so the responses weren't correct. Secondly, even if there were correct responses, at times the level of detail or preciseness delivered towards the subject of many of the user's inquiries felt stone chiseled and repetitious. Lastly, we have to be concerned about not only what the user reads and if it is the correct information, but we also have to be concerned about how the user feels about the information they are receiving. At the very least you can give the user the information that they require. The best is to give them the information in an empathetic way aligned with retail customer service best practices.

/old maintenance site

"Intent refinement for AI isn't a visual contention. It is when you realize that the user has something in mind as for a typical response in contextual language. You need to have your usability wrapped around that concept or pattern of awareness already. It's up to you to think ahead like chess and respond to that idea. "

THE PROCESS

1. User Research: The Big Data Gold Mine

The process of improving an artificial intelligence component towards an improved contextual response calls for a ton of data.  Our main goal with the data is to see if we can recognize patterns between what users are typically looking for and also how they take in the response from our attempts in delivery. Luckily, we designed our chat environment to produce a daily log of all input/output interaction and since the launch has collected over 40,000 lines of raw data. This 40,000 lines of accrued data became a tell-all in everything we were looking for from the human-computer interaction. We could see exactly what the user was searching for. We can also see exactly where Watson Assistant was inaccurate and how that affected the user. With these three main data points, we could devise a plan for an improved dialogue flow and even throw in customer service and factors of a focused personalization. These logs became our ground truth and, through analysis, we could use these as a means for the customer to tell us what they wanted to have as an ultimate experience. 

/User Research

2. Reconfiguring IBM Watson

After scanning the user inputs we noticed a strong pattern that many user inputs either were missed because they were never considered or perhaps got shifted around due to wording associations. So we isolated the wording in IBM Watson assistant by broadening the intent libraries with a more focused and targeted inclination to reach beyond the intents we were expecting. We simply utilized the customers typed phrases and categorized them to fit within the guidelines of the services users were trying to receive. By utilizing the user's attempts to communicate with the chatbot as feedback and data within itself drives 

/IBM Watson API

3. Fine Tuning K-Fold

We performed a basic unit Kfold test using Python to see if our efforts in beefing up the intent library actually did justice. In lamens terms, Kfold testing works by removing a phrase from its designated intent category. If the intent, based on the wording itself, pulls the correct intent then our wording is considered successful in that regard. If it pulls from another intent it means we have to adjust some of the entries in both intent libraries to get a more accurate response. The Kfold results can also be utilized to measure overall accuracy as they can reveal, collectively, how many phrases hit their target or not. It is important to do this step many times to actually pull as much accuracy as you can to get a more tuned result. 

/K-Fold Diagram

4. New Dynamic Dialogue

Once we have determined a desired and acceptable means of success we then can guide the user down several paths in order to contextually deliver their desired information. IBM Watson is comprised of static nodes which is why we get a repetitious answer.  Our solution for that is to pragmatically or reasonably navigate the user through several possible outcomes. This, in turn, allows the conversation to feel more on the human tail side as humans never usually say the same thing the same exact way twice. This pretense becomes achievable only if we think ahead and consider the possibilities and the patterns to program the reason and relative responses.

/Flow Chart Example

THE ANALYTICS

The Results

The results for completing 

• Intent Accuracy Increase to 80% (+10%)
• More Dynamic Range of Responses
• Retail Customer Service Feel
• Information Gateway
  

/analytics dashboard