Integrating a python Tensorflow chatbot model into a Xamarin app

Question

I'm using Xamarin Forms to create a chatbot-like messaging app and I've created a simple chatbot model using TensorFlow and python. The model executes as a console app and after some testing, I've been able to run the python chatbot script in C# console using Pythonnet. I would now like to integrate it into the Xamarin app wherein the users text is taken in as the input and the output is from the chatbot. How would I use this python file as a backend?

Chatbot.py

#MODULE IMPORTS
import nltk
from nltk.stem.lancaster import LancasterStemmer
import numpy
import tflearn
import tensorflow
import random
import json
import pickle
stemmer = LancasterStemmer()

with open("C:/Users/PAVILION/Desktop/Trial3/Conversations/intents.json") as file:
    data = json.load(file)

try:
    with open("data.pickle", "rb") as f:
        words, labels, training, output = pickle.load(f)

except:
    words = []
    labels = []
    docs_x = []
    docs_y = []

    for intent in data["intents"]:
        for pattern in intent["patterns"]:
            #STEMMING OF WORDS
            stemmedWords = nltk.word_tokenize(pattern)
            words.extend(stemmedWords)
            docs_x.append(stemmedWords)
            docs_y.append(intent["tag"])

        if intent["tag"] not in labels:
            labels.append(intent["tag"])

    words = [stemmer.stem(w.lower()) for w in words if w != "?"] 
    words = sorted(list(set(words)))
    labels = sorted(labels)


    training = []
    output = []
    out_empty = [0 for _ in range(len(labels))]

    for x, doc in enumerate(docs_x):
        bag = []
        stemmedWords = [stemmer.stem(w) for w in doc]

        for w in words:
            if w in stemmedWords:
                bag.append(1)
            else:
                bag.append(0)

        output_row = out_empty[:]
        output_row[labels.index(docs_y[x])] = 1

        training.append(bag)
        output.append(output_row)

    training = numpy.array(training)
    output = numpy.array(output)

    with open("data.pickle", "wb") as f:
        pickle.dump((words, labels, training, output), f)


tensorflow.reset_default_graph()

net = tflearn.input_data(shape = [None, len(training[0])])
net = tflearn.fully_connected(net, 8)
net = tflearn.fully_connected(net, 8)
net = tflearn.fully_connected(net, len(output[0]), activation="softmax")
net = tflearn.regression(net)

model = tflearn.DNN(net)

try:
    model.load("model.tflearn")

except:
    model.fit(training, output, n_epoch=1000, batch_size=8, show_metric=True)
    model.save("model.tflearn")

def bagOfWords(s, words):
    bag = [0 for _ in range(len(words))]
    s_words = nltk.word_tokenize(s)
    s_words = [stemmer.stem(word.lower()) for word in s_words]

    for s2 in s_words:
        for i, w in enumerate(words):
            if w == s2:
                bag[i] = (1)
    return numpy.array(bag)

def chat():
    print("Start talking with the bot")
    while True:
        userInput = input("You: ")
        if userInput.lower() == "quit":
            break
        results = model.predict([bagOfWords(userInput, words)])[0]
        results_index = numpy.argmax(results)
        tag = labels[results_index]

        if results[results_index] > 0.6:
            for tg in data["intents"]:
                if tg["tag"] == tag:
                    responses = tg["responses"]

            print(random.choice(responses))

        else:
            print("Dont understand")

chat()

Chatbot.xaml in Xamarin

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:d="http://xamarin.com/schemas/2014/forms/design"
             xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
             xmlns:controls="clr-namespace:BluePillApp.Controls"
             xmlns:renderers="clr-namespace:BluePillApp.CustomRenderers"
             xmlns:local="clr-namespace:BluePillApp"
             xmlns:helpers="clr-namespace:BluePillApp.Helpers"
             xmlns:xamEffects="clr-namespace:XamEffects;assembly=XamEffects"
             xmlns:pancake="clr-namespace:Xamarin.Forms.PancakeView;assembly=Xamarin.Forms.PancakeView"
             mc:Ignorable="d"
             x:Class="BluePillApp.Views.ChatbotPage">

    <ContentPage.Resources>
        <ResourceDictionary>
            <helpers:TemplateSelector x:Key="MessageTemplateSelector"/>
        </ResourceDictionary>
    </ContentPage.Resources>

    <Grid RowSpacing="0" ColumnSpacing="0">
        <Grid.RowDefinitions>
            <RowDefinition Height="Auto"/>
            <RowDefinition Height="*"/>
            <RowDefinition Height="1"/>
            <RowDefinition Height="Auto"/>
        </Grid.RowDefinitions>

        <!--Chat Views-->

        <!--Chat Messages Area-->
        <ListView Grid.Row="1" 
                 ItemTemplate="{StaticResource MessageTemplateSelector}" 
                 ItemsSource="{Binding Messages}" 
                 Margin="0" 
                 HasUnevenRows="True" 
                 VerticalOptions="FillAndExpand" 
                 SeparatorColor="Transparent" >
        </ListView>

        <!--A simple separater-->
        <BoxView BackgroundColor="LightGray"
                 HorizontalOptions="FillAndExpand"
                 Grid.Row="2"/>

        <!--Chat Entry and Send Button-->
        <StackLayout Orientation="Horizontal" 
                     Margin="0" 
                     Grid.Row="3" 
                     Padding="5,10,10,10">

            <!--Chat Entry-->
            <renderers:RoundedEntry Text="{Binding TextToSend}" 
                                    x:Name="ChatEntry"
                                    Placeholder="Type your message"
                                    CornerRadius="30"
                                    BackColor="White"
                                    BackgroundColor="Transparent"
                                    BorderColor="#999999"
                                    BorderWidth="5"
                                    FontSize="16"
                                    PlaceholderColor="#b3b3b3"
                                    Keyboard="Chat"
                                    WidthRequest="280"/>

            <!--Send Button-->
            <pancake:PancakeView BackgroundColor="#0f8df4"
                                 CornerRadius="100"
                                 xamEffects:TouchEffect.Color="White"
                                 xamEffects:Commands.Tap="{Binding SendCommand}"
                                 HeightRequest="47"
                                 WidthRequest="47"
                                 HorizontalOptions="EndAndExpand">

                <Image Source="sendarrow1.png"
                       HeightRequest="30"
                       WidthRequest="30"
                       Margin="5,0,0,0"
                       HorizontalOptions="Center"
                       VerticalOptions="Center"/>
            </pancake:PancakeView>
        </StackLayout>



        <!--Chatbot Header and Back Button-->
        <pancake:PancakeView BackgroundColor="#f7f7f7" Grid.Row="0" HasShadow="True" Padding="0,10,0,10">
            <StackLayout Orientation="Horizontal">

                <!--Back Arrow Button-->
                <renderers:FontAwesomeIcon Text="{x:Static helpers:IconsFA.BackArrow}" 
                                           HorizontalOptions="Start"
                                           TextColor="Black"
                                           FontSize="20"
                                           VerticalTextAlignment="Center"
                                           Margin="15,0,0,0">
                    <Label.GestureRecognizers>
                        <TapGestureRecognizer Tapped="BacktoMain_Button"/>
                    </Label.GestureRecognizers>
                </renderers:FontAwesomeIcon>

                <!--Fleming Image-->
                <StackLayout Orientation="Horizontal">
                    <Frame BackgroundColor="Gray" 
                           Padding="0" 
                           CornerRadius="100"
                           HasShadow="False"
                           IsClippedToBounds="True"
                           HeightRequest="40"
                           WidthRequest="40">

                        <Image HorizontalOptions="Center"
                               VerticalOptions="Center"/>
                    </Frame>

                    <!--Fleming Text-->
                    <Label Text="Fleming"
                           TextColor="Black"
                           FontAttributes="Bold"
                           FontSize="18"
                           VerticalOptions="Center"/>
                </StackLayout>
            </StackLayout>
        </pancake:PancakeView>
    </Grid>
</ContentPage>

Chatbot.xaml ViewModel

 public class ChatbotPageViewModel : INotifyPropertyChanged
    {
        public event PropertyChangedEventHandler PropertyChanged;

        /// <summary>
        /// A collection.list of chat message items
        /// </summary>
        public ObservableCollection<ChatMessageModel> Messages { get; set; } = new ObservableCollection<ChatMessageModel>();

        /// <summary>
        /// The text that the user inputs
        /// </summary>
        public string TextToSend { get; set; }

        /// <summary>
        /// A command for sending the users messages
        /// </summary>
        public ICommand SendCommand { get; set; }


        /// <summary>
        /// ChatPageViewModel Constructor
        /// </summary>
        public ChatbotPageViewModel()
        {
            Messages.Add(new ChatMessageModel() { Text = "Hi" });
            Messages.Add(new ChatMessageModel() { Text = "How are you?" });

            SendCommand = new RelayCommand(Send);
        }

        /// <summary>
        /// This function sends a message
        /// </summary>
        private void Send()
        {
            if (!string.IsNullOrEmpty(TextToSend))
            {
                //This adds the following to the messages collection
                Messages.Add(new ChatMessageModel() { Text = TextToSend, User = App.User});

                if(TextToSend == "Hey")
                {
                    Messages.Add(new ChatMessageModel() { Text = "Hey yourself"});
                }
            }
        }
    }

Sample of the full Xamarin Chatbot app I'm developing

two actually, python and C#. I want to make a Xamarin chatbot app and need it to utilize TensorFlow machine learning, so Python was the only feasible choice. Theres no real machine learning solution for C# except ML.Net which is still in a very early stage. — Draxalot2, Nov 23 '19 at 00:19
the 3rd was that you are most likely compiling this to something else like IOS, or andriod. TensorFlow isnt a python thing, its just a thing you can use it anywhere on any platform — TheGeneral, Nov 23 '19 at 00:22
With Xamarin I believe it would all just be a single C# codebase. And I've done some searching but there's no .NET version of Tensorflow that can do all the things I need it to — Draxalot2, Nov 23 '19 at 01:20
@TheGeneral Is there any way you can think of achieving this? — Draxalot2, Nov 24 '19 at 18:11

score 0 · Answer 1 · answered Nov 25 '19 at 14:32

0

There is some misunderstanding of concepts. You don't need the python file been integrated into XF project. You use Python as a language and TensorFlow as a framework to get ML model. A model is a result of the training process and you could use it in other places like mobile platforms.

For using models on Android you could try to use TensorFlow Lite

For using models in iOS you should convert default format of TF in CoreML format (.mlmodel)

After that, you have to integrate these models into Xamarin.iOS and Xamarin.Android projects

answered Nov 25 '19 at 14:32

Denis Gordin

892
10
22

I see, so what file format would be used for each platform? Because in my chatbot folder, the model produced a .meta, .index, .pickle and a .data-00000-of-00001 file – Draxalot2 Nov 25 '19 at 22:53
@Draxalot2 default format for TF models is .pb. I see that you use tflearn for training. I'm not familiar with either tflearn or even TF but you could find some solutions for the issue here https://github.com/tflearn/tflearn/issues/964 – Denis Gordin Nov 25 '19 at 23:42
okay, do you have any knowledge/experience on integrating said model with Xamarin.Android and Xamarin.iOS? – Draxalot2 Nov 25 '19 at 23:45
For Xamarin projects - no. But even official docs for Xamarin.iOS [could](https://learn.microsoft.com/en-us/xamarin/ios/platform/introduction-to-ios11/coreml) help if you spend a bit more time for googling it. – Denis Gordin Nov 26 '19 at 01:07

Integrating a python Tensorflow chatbot model into a Xamarin app

1 Answers1