2

Hey I am trying to train my chatbot with existing corpuses using chatterbot but I found that my chatbot cannot be trained. It seems to get stuck when it comes to the line bot.train(data). Here is my code:

from chatterbot import ChatBot
from chatterbot.trainers import ListTrainer
import os

bot = ChatBot('ChatBot')
bot.set_trainer(ListTrainer)
new = []

for files in os.listdir('/Users/christinalai/Desktop/corpus'):
    data = open('/Users/christinalai/Desktop/corpus/' + files, mode='r',errors="ignore").readlines()
    for line in data:
        new.append(line)

bot.train(new)

while True:
    message = input("You: ")
    if message.strip() not in message_list:
        reply = bot.get_response(message)
        print("ChatBot: ", reply)

Thanks for looking at my question.

Christina Lai
  • 33
  • 3
  • 1
    What do you mean by "stuck"? It might just be taking a long time to train with the corpora you used. – Scratch'N'Purr Jun 26 '18 at 08:27
  • I meant that the program is not going into the while loop and it is not finished either. When I made the corpus smaller, to only 2 small corpuses, the errors are shown below: Traceback (most recent call last): ... sqlite3.OperationalError: database is locked ... The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/Users/christinalai/PycharmProjects/chatterbot/chatbot.py", line 17, in bot.train(new) ... – Christina Lai Jun 26 '18 at 14:04
  • Well, it looks like you have some uncommitted transactions in the database that your chatbot is writing to. This could explain why the script is stuck because the db is still writing records from either a previous training session or another process. – Scratch'N'Purr Jun 26 '18 at 14:25
  • I am sorry I am new to chatterbot. What do you mean by "uncommited transactions in the database"? What is the database exactly and where can I find it? – Christina Lai Jun 26 '18 at 14:46
  • Take a look [here](https://www.sqlite.org/lockingv3.html) on how writing to database works. You're most likely using a SQLite since it is the default database for chatterbot. The file will be called `db.sqlite3` and it will most likely reside in the directory where your bot is. – Scratch'N'Purr Jun 27 '18 at 06:54
  • Just delete the db.sqlite3 file from your project directory and execute the code again. – Mufeed Jun 27 '18 at 13:21
  • Thank you so much!! It works after I deleted the db.sqlite3 database. There are no errors being printed when I ran the program but the training session seems to take forever... I collected corpuses of about the size of 2 GB and it takes so much time to train the chatbot, though it works just fine using a small corpus. Still, thanks a lot :) – Christina Lai Jun 30 '18 at 06:17

1 Answers1

0

You can use this function to retrieve your copus content as a list. And then you can train chatbot as usual.

def delcha(path):
    with open(text_file,encoding='utf-8') as file:
        list_line_file = file.readlines()
    for ans in list_line_file:
        if '\n' in ans:
            edit_ans = ans[0:len(ans)-1]
            index = list_line_file.index(ans)
            list_line_file.pop(index)
            list_line_file.insert(index, edit_ans)
    return list_line_file

With this function you simply pass the parameter to the path to your file, and it will return a List of lines in that file.

AXin
  • 23
  • 1
  • 6