Why We Replaced An App with a Chatbot

Alex Ottenwess


This blog was co-authored with Fiona Campbell, Matthew Ehinger, and Cassie Free.

Last month, Solstice FWD, Solstice’s annual innovation summit, brought together 400 industry leaders to discuss the future of digital customer experience and to witness firsthand the the newest interactive experiences coming out of Solstice Labs.

Amongst the many ambitions for this year’s event, the Solstice Labs team set out to leverage the power of Artificial Intelligence and the latest in conversational UX to elevate the typical conference experience by providing attendees with more engaging, personalized and contextual interactions. To handle both simpler commands such as answering logistical questions that might arise throughout the day and more advanced thinking such as making individual, contextual networking recommendations. Luckily, we had the perfect platform -- and personality -- in mind for the job.

Diving into the world of conversational UX and artificial intelligence, we created our very own AI platform, Siena. Powering Solstice FWD, attendees could talk with Siena through a variety of interactive experiences. One experience in particular was a chatbot called “Solstice Siena.” While there were many routes that could have been chosen to showcase this technology, we set our sights on killing the typical, boring conference app in favor of smarter, conversational interfaces in Siena..

About Siena

Siena_Chatbot1.pngRather than asking attendees to download another app for a one-off event, Siena was served up to conference attendees through Facebook Messenger, the largest and most ubiquitous chat channel in the world.

Through simple chats, Siena answered attendees questions around speakers, Solstice Labs experiences, the agenda, and various venue locations such as the bathroom and registration. Working in conjunction with attendees’ beacons and the Solstice location services platform, Siena was able to identify the location of a given attendee at any moment, and used this information to provide a more personal experience. Finally, Siena was also designed as a personal networking assistant. Upon a user’s request to network with their peers at the event, she emailed them headshots, contact information, and LinkedIn profiles of other attendees they may be interested in meeting.

In the end, Siena provided a personal experience for every conference attendee and the positive response was overwhelming.

Now let’s dig into how we built Siena chatbot and the lessons learned...


Architecture Decisions

When deciding how to build a chatbot, there were a few things you will want to consider. In this instance, we wanted multiple chat or voice clients like Facebook Messenger, Slack, and even SMS to have the ability to tap into a single, unified personality: Siena. Secondly, we wanted to avoid getting locked into one NLP (Natural Language Processing) service. We wanted the flexibility to integrate with new NLP technologies later on. Finally, we wanted to build a sustainable and scalable Conversational UX platform whose use-cases apply to a myriad of industries.

We accomplished these goals by splitting our system into three main components: a client chat layer, an NLP layer, and a Response Generation layer. The combination of the NLP layer and Response Generation layer made up what we called “Siena AI”, the intelligent system powering our chatbot. By having one entry-point into Siena AI, we were able to build, test, and iterate on the Facebook Messenger client first, and later plug into an SMS client that took on the exact same Siena personality with limited additional work.

Separating the client chat layer from the NLP layer was also an important design decision, as it gave us the ability to try out multiple other NLP services (e.g. api.ai or msg.ai) without affecting the rest of our system. As NLP services inevitably become more abundant and more powerful, it will be necessary to swap out old NLP implementations for newer, smarter implementations.

Finally, we found that a serverless architecture deployed using AWS API Gateway and Lambda would be the ideal approach to achieve our third goal: scale. Unfortunately, due to some JavaScript ES6 features required by the latest version of wit.ai, we were unable to easily deploy our client and NLP layers to AWS Lambda, and instead hosted the servers on Heroku. Nonetheless, our 3-tiered architecture allowed us to build a sustainable, extensible, and scalable chatbot in a couple of months.Siena_arch.png

Pitfalls and Problems Encountered

As we developed a more and more advanced bot we naturally ran into issues. We lacked a fully isolated testing environment; Wit.ai proved to be unreliable for the caliber of usage we needed. Whenever you employ a third party service in an app you are developing, you accept a risk that the service may not be reliable, and may not behave exactly as you require. Although Wit.ai is an amazing platform - free, intuitive, and powerful - like any NLP service, it has inherent downfalls. Specifically, we ran into issues training Wit. We found that after a training spree Wit could become “overwhelmed”, and return vastly incorrect interpretations of queries. Eventually, we established the policy that only one person should even have the Wit.ai console up at a time, as we noticed multiple trainers would drastically dampen Wit’s performance.

The other major issue we encountered when building our bot was defining the scope and direction. As with any conversational tool, the purpose and personality need to be clearly defined because the possibilities are endless. What personality should the bot embody? What features do we need to include? How do we empower the user to utilize the many features we make? We learned from this experience the importance of user research, QA and user testing, and clearly defining the purpose of the bot. The possible topics for a conversational platform are endless so the purpose of the bot needs to be clearly defined for the user.

Lessons Learned and Takeaways

Throughout this experience, from development to end-user experience, two things really stuck out. First, we learned that users expect a chatbot to have human-like qualities. This creates a significant technical challenge -- building a personality, not just a command line. Secondly, user-experience improves dramatically when the bot provides guidance and “hand-holds” the user through conversation, rather than simply responding when asked a question.

For Siena 2.0, we have identified opportunities to grow the Siena platform based on these lessons learned. First, leveraging Api.ai NLP service instead of Wit.ai. Wit.ai was great for a simple chatbot POC, but doesn’t have built-in capabilities for building in a robust “personality” of a bot. Api.ai, however, has an expansive set of “pre-trained knowledge packs” called Domains that allow developers to easily handle non-business-logic related queries such as Small Talk, Weather, News, Sports, etc. With Api.ai, you can easily add these Domains into your project, and customize how your bot responds to questions like “Hey, what’s up?” or “What do you know?” without any additional model training. This is a very powerful feature that can add human-like qualities to your bot very quickly. Finally, in order to provide guidance to the user, integrating clickable “Quick Replies” into our chatbot will provide the ability to “hand hold” the user through conversation, prompting them with relevant next steps. Nudging the user through “happy path” conversations makes for a much more pleasant interaction, while training the user on the bot’s capabilities.

In the end, user experience is what determines the success or failure of a conversational interface. In order to achieve an enjoyable user experience, the right combination of technologies needs to be applied to create a robust and responsive chatbot that meets the user's expectations. Even though the current NLP services are still in their infancy, we believe there is enough capability to create an enjoyable conversational UX platform that is applicable to a wide array of industries and use-cases.


Interested in talking with our team about integrating bots into your digital strategy? Contact us today.

For more information about bots and conversational UX check out Paul Heckel’s research paper “There’s a Bot for That: The new mobile strategy” at: http://www.solstice.com/bots-the-new-mobile-strategy.