Extending chatbot platform with Microsoft Azure

July 31, 2018
bot-framework azure-resource-manager

Together with Wingbot.ai and Česká spořitelna we have extended existing chatbot framework with support for Microsoft Azure Bot Service and deployment to Microsoft Azure.

Motivation

There are many chatbot frameworks out there. And although Microsoft’s Bot Builder SDK is the recommended way of working with Azure Bot Service, not everyone is able (or keen) to use it. In our case the customer has picked Wingbot.ai as their bot building platform and it was up to some engineering magic to add Azure support to it.

If you’re not familiar with Wingbot, this is how David, its creator, describes it in combination with Azure:

Wingbot.ai platform stands on a set of opensource Node.js modules connected with bot designer service. Modularity opens the way to integrate with Azure services and Azure cloud provides the environment that meets banking security standards.

Wingbot.ai designer service

Wingbot.ai-powered chatbot running on Azure

Wingbot is currently used by Česká spořitelna for several chatbots, some of them will become public soon.

How to use Wingbot.ai with Azure

Prerequisites:

  1. Active Azure account: https://azure.microsoft.com

  2. Node.js / NPM

  3. Azure CLI 2.0 (cross-platform)

  4. Sign into the CLI and choose correct subscription:

    az login
    az account set --subscription "<subscription name or id>"
    
  5. Register an app for your bot using the App Registration portal - get Application ID and Password

  6. If you want to use the Wingbot.ai conversation designer, you also need a Wingbot account.

Create and deploy:

  1. Install wingbot-cli

    At the time of this writing, Azure support is not part of the NPM package. To use it, clone the CLI and use local version.

    git clone https://github.com/wingbotai/wingbot-cli
    cd wingbot-cli
    npm install
    node ./bin/wingbot.js init
    
  2. Create new bot using the Wingbot CLI: wingbot init

  3. Select Azure Functions or Azure Web Apps as the infrastructure platform.

  4. Select Azure Bot Service as bot platform.

  5. Go through the rest of the wizard.

  6. Run deploy script:

    cd bin
    ./deploy.cmd
    

Detailed solution

We didn’t want to just make Wingbot run on Azure, our goal was to make Azure Bot Service one of its core platforms with App Service hosting, Bot Framework emulator support, third-party channels and seamless deployment.

There are currently two parts to this solution:

  1. Azure Resource Manager templates with deployment scripts,
  2. Wingbot.ai connector for Azure Bot Service.

Deployment templates for Azure

While it is possible to create BAT/SH/PS (whichever you prefer) script to deploy the whole infrastructure to Azure step-by-step and then upload bot code, I decided to use Azure Resource Manager template as the preferred method to provision all necessary resources (App Service, Bot Service, database etc.) and combine it with Azure CLI in order to upload ZIPped code.

Wingbot CLI

Wingbot already has a neat command-line utility to generate boilerplate project. It’s based on commander and handlebars. During our journey to Azure, we enhanced it with support for multiple Azure services:

Wingbot CLI with support for Azure services

https://github.com/wingbotai/wingbot-cli

CLI generates:

  1. bot project, deployable to chosen hosting platform (Functions / Web Apps),
  2. resource manager template JSON file,
  3. resource manager parameters JSON file (with values from CLI),
  4. deployment script for chosen hosting platform (Functions / Web Apps).

Bot project

To be compatible with Azure Functions, the original bot project, which was based on Express, had to be adjusted to work with Functions Runtime.

// standard HTTP-triggered Azure Function
module.exports = async function (context, req) {
    if (req.method === 'GET') {
        return {
            status: 200,
            body: '{"message":"RUNNING"}'
        };
    }

    const { botToken = null, senderId = null } = req.query || {};

    if (botToken && senderId) {
        const tokenObj = await botTokenStorage.findByToken(botToken);

        if (tokenObj.senderId !== req.body.sender.id) {
            return {
                status: 401,
                headers: { 'content-type': 'text/plain' },
                body: 'Unauthorized'
            };
        }

        await botService.processMessage(req.body, tokenObj.senderId, tokenObj.pageId);
    } else {
        await botService.verifyRequest(req.body, req.headers);
        await botService.processEvent(req.body);
    }

    return {
        status: 200,
        body: '{"message":"OK"}'
    };
};

This code is a result of translating handlebars template with parameters from CLI. Here’s a glimpse of part of the template to give you an idea:

...
{{#if azureServerless}}
{{#if botService}}
module.exports = async function (context, req) {
    if (req.method === 'GET') {
        return {
            status: 200,
            body: '{"message":"RUNNING"}'
        };
    }

    const { botToken = null, senderId = null } = req.query || {};

    if (botToken && senderId) {
        const tokenObj = await botTokenStorage.findByToken(botToken);

        if (tokenObj.senderId !== req.body.sender.id) {
            return {
                status: 401,
                headers: { 'content-type': 'text/plain' },
                body: 'Unauthorized'
            };
        }

        await botService.processMessage(req.body, tokenObj.senderId, tokenObj.pageId);
    } else {
        await botService.verifyRequest(req.body, req.headers);
        await botService.processEvent(req.body);
    }

    return {
        status: 200,
        body: '{"message":"OK"}'
    };
};
{{/if}}
{{/if}}
{{#if expressOrAppService}}
{{#if messenger}}
module.exports = [
    express.json({
        verify: (req, res, buf) =>
            facebook.verifyRequest(buf, req.headers)
    }),
    wrapRoute(async (req, res) => {
        if (req.method === 'GET') {
            const body = await facebook.verifyWebhook(req.query);
            res.send(body);
            return;
        }

        await facebook.processEvent(req.body);
        res.send('OK');
    })
];
{{/if}}
{{#if botService}}
module.exports = [
    express.json(),
    wrapRoute(async (req, res) => {
        if (req.method === 'GET') {
            res.send('RUNNING');
            return;
        }

        await botService.verifyRequest(req.body, req.headers);
        await botService.processEvent(req.body);
        res.send('OK');
    })
];
{{/if}}
{{/if}}
...

For Web Apps hosting, no changes on the application side were needed - Azure Web Apps support Express without modification.

ARM template

Similar to bot code, the Resource Manager template is generated dynamically, so it’s no surprise that it relies on handlebars too.

Parameters:

Name Description Note
appName Web App / Function App name where the bot will be hosted.
botSku SKU of Bot Service (F0 or S1)
botName Name of the bot for Bot Service Important: Bot name must not use MS reserved words (such as “skype”, “cortana”, “webapp” etc.).
In such case the deployment fails with message: Id: Id is already in use...
botDisplayName Display name for Bot Service
botAppId Application ID for Bot Service Required. Has to be provided, otherwise provisioning fails.
botAppPassword Password of the bot application Not required for provisioning, but the bot will not work without it.
cosmosDbName Name of Cosmos DB database If present, new Cosmos DB resource will be created and cosmosDbConnectionString will be filled automatically.
If omitted, no database will be created and cosmosDbConnectionString has to be provided.
cosmosDbConnectionString Connection string to Cosmos DB database Ignored when cosmosDbName provided.
Can be also a connection string to Mongo DB instance.

Resources created by the template:

Resource Purpose
Application Insights Connected to Bot Service for telemetry.
Storage Account For App Service.
Cosmos DB Default throughtput 1000 RU/s.
Created only if Database name was provided when generating project.
Function App Consumption App Service Plan.
If Wingbot type is Azure Functions.
Web App Free App Service Plan.
If Wingbot type is Azure App Service.
Bot Service With Facebook, Web Chat and DirectLine channels enabled.
If selected when generating project.
Requires Application ID from Microsoft App Registration Portal.

Notes:

There are several significant application settings in the template.

It’s important to change Node.js version running on the App Service. Wingbot uses modern JavaScript features (async/await etc.) and doesn’t work on default Node.js version.

{
    "name": "WEBSITE_NODE_DEFAULT_VERSION",
    "value": "8.11.1"
},

Also, use v2 (currently in beta) runtime of Azure Functions to be able to change Node.js version.

{
    "name": "FUNCTIONS_EXTENSION_VERSION",
    "value": "beta"
},

Build the app after ZIP deployment, so that you don’t have to provide node_modules.

{
    "name": "SCM_DO_BUILD_DURING_DEPLOYMENT",
    "value": "true"
}

Create Bot Service resource with multiple channels and Application Insights integration.

{
    "type": "Microsoft.BotService/botServices",
    "sku": {
        "name": "[parameters('botSku')]"
    },
    "kind": "bot",
    "name": "[parameters('botName')]",
    "apiVersion": "2017-12-01",
    "location": "global",
    "properties": {
        "name": "[parameters('botName')]",
        "displayName": "[parameters('botDisplayName')]",
        "endpoint": "[concat('https://', variables('appName'), '.azurewebsites.net/bot')]",
        "enabledChannels": [
            "webchat",
            "directline",
            "facebook"
        ],
        "configuredChannels": [
            "webchat",
            "directline",
            "facebook"
        ],
        "msaAppId": "[parameters('botAppId')]",
        "developerAppInsightKey": "[reference(resourceId('Microsoft.Insights/components', variables('insightsName')), '2014-04-01').InstrumentationKey]"
    },
    "dependsOn": [
        "[resourceId('Microsoft.Web/sites', variables('appName'))]"
    ]
}

ZIP deploy

Azure App Service supports multiple ways to deploy code. I figured that to allow fast start it would be best to use quite new Kudu feature called ZIP deploy. As the name suggests it allows developers to provide their whole application as a ZIP file.

Note: Everything in that file is deployed, everything not in the file is deleted from the server.

By default, ZIP deploy expects to find in the package everything necessary to run the app. For Node.js that means even the node_modules folder. I wanted the app to be built on deploy, though, so I have included the SCM_DO_BUILD_DURING_DEPLOYMENT App Service parameter.

Creation of the ZIP file is handled by npm-pack-zip package. It’s similar to npm pack, but creates ZIP file and doesn’t put files into subfolder inside the archive. I have prepared a build task to pack the application:

npm run pack

And because there are several files which do not need to be deployed, I’ve also added the .npmignore file:

/deployment
/bin
/test
jsconfig.json
.eslintrc
mocha.opts
local.settings.json

Deployment script

Wingbot CLI generator for Azure currently adds one file to the bin folder: deploy.cmd (Bash and PowerShell are planned). The idea is to call this script on first deployment, as it provisions the whole Azure infrastructure and uploads application ZIP package.

@echo off
echo Installing NPM packages...
call npm install

if exist "..\mybot.zip" (
    echo ZIP package exists, deleting.
    del "..\mybot.zip"
)

echo Creating ZIP package...
call npm run pack

echo Creating resource group...
call az group create -n "mybot-rg" -l "northeurope"

echo Creating resources...
call az group deployment create -g "mybot-rg" --template-file ..\deployment\template.json --parameters ..\deployment\parameters.json

echo Deploying...
call az functionapp deployment source config-zip -g "mybot-rg" -n "mybot" --src "..\mybot.zip"

echo Done.

It’s obvious now why one of the requirements is to have Azure CLI installed.

Connector for Azure Bot Service

This part was mostly done by David Menger, who refactored Wingbot.ai to be modular and implemented Bot Service APIs into a special module wingbot-botservice. The whole framework is open-source, so let’s just highlight some of the most important parts you have to think of when implementing your own Bot Service SDK.

Documentation

Azure Bot Service, like the rest of Microsoft’s cloud platform, provides REST APIs as an alternative to SDK libraries. To find the official documentation for for this API you need to search for Bot Connector REST API.

Beware, there’s also Bot Builder REST API, which is not what you need in this case.

Even though the documentation is not completely missing, there are some parts which are not documented that well. That’s when the official SDK comes handy, especially when trying to understand the data model used by the connector service.

Transformation

Wingbot was originally implemented with Facebook Messenger messages as a priority, so it works natively with Facebook events. Bot Service layer is basically “just” a transformation from one message form to another - if particular event comes from Messenger, it’s passed through directly, otherwise it’s transformed.

Authentication

Coming soon…

Emulator

Coming soon…

Next steps

With the bot runnig and deployable, what’s next? One significant part of Wingbot is natural language processing (NLP) and during our one week hack we didn’t get to implement connector for Microsoft LUIS service. So that’s definitely coming, along with minor fixes and tweaks to finalize full support for Bot Service.

comments powered by Disqus