Bot dialogs unit testing using Stories concept

Bot development using Microsoft Bot Framework is easy and enjoyable. It doesn’t take a lot of time to setup basic bot you can talk to. However, things are getting more complicated if you want to develop high-quality, unit-tested bot code for your customer. Since code quality is very important value for my organization, I was asked to develop a testing framework for bots, which would make dialog unit testing easy and intuitive.

Bot dialog story

My task was easier thanks to an existing library developed by my company colleague – Sławomir Kopacz. It’s called Bot.Test.Stories and it’s available on GitHub and NuGet. It allows to record a conversation between the user and bot dialog in a very intuitive way.

Objectivity.Bot.Tests.Stories.Xunit library

The unit testing framework I’ve created allows to take an advantage of Stories concept to write unit tests for your dialogs. It’s available on NuGet and it’s source code is available on GitHub. With this library unit testing is very simple – you just need to setup a class deriving a generic DialogUnitTestBase type, write a test story and play it:

using System.Threading.Tasks;
using Dialogs;
using global::Xunit;
using Recorder;

public class SumDialogTests : DialogUnitTestBase
{
  [Fact]
  public async Task RecordedStory_PlayStoryIsCalled_DialogDone()
  {
    var story = StoryRecorder
    .Record()
    .Bot.Says("Hi! What can I do for you?")
    .User.Says("When is my next meeting?")
    .Bot.Says("Your next meeting is today at 1 PM.")
    .Rewind();

    await this.Play(story);
  }
}

The framework enhances the Stories concept with some additional ending variants to allow the developer verify the way the tested dialog was finished:

  • DialogDone() – verifies the dialog was done (context.Done() was called) – ignores the dialog result:
var story = StoryRecorder
    .Record()
    .Bot.Says("Hi! What can I do for you?")
    .User.Says("Is John Smith available today?")
    .Bot.Says("He's out of office today.")
    .Bot.Says("Can I help you with anything else?")
    .User.Says("No, thanks.")
    .Bot.Says("Good bye!")
    .DialogDone();

await this.Play(story);
  • DialogDoneWithResult(predicate) – verifies the dialog was done and checks the result predicate:
var story = StoryRecorder
    .Record()
    .User.Says("What's my company ID number?")
    .DialogDoneWithResult<string>(id => id == "5213462");

await this.Play(story);
  • DialogFailed() – verifies the dialog failed (context.Fail() was called) – ignores the exception  thrown by the dialog:
var story = StoryRecorder
    .Record()
    .User.Says("Create a ticket for me")
    .Bot.Says("Please select one of ticket types:")
    .Bot.Says("[1] - IT, [2] - Administration")
    // choice out of range
    .User.Says("3")
    .DialogFailed();

await this.Play(story);
  • DialogFailedWithExceptionOfType() – verifies the dialog failed with the exception type equal to provided in generic attribute:
    var story = StoryRecorder
    .Record()
    .Bot.Says("Good evening, sir! How can I help you?")
    .User.Says("I'd like to book a table")
    .Bot.Says("For how many people?")
    // expected a number
    .User.Says("for two")
    .DialogFailedWithExceptionOfType<FormatException>();

await this.Play(story);

More examples can be found on GitHub in samples project.

Leave a Reply

Your email address will not be published. Required fields are marked *