Implement Dynamic Bayesian Network in Unity3d using Bayes server c# library

Question

I am trying to create a dynamic Bayesian network for parameter learning using the Bayes server in C# in my Unity game. The implementation is based on this article.

A brief explanation of the model shown in the figure below: When a player starts playing the level I assign them an initial probability of 0.5 that they already know the stuff that they are learning which is represented as Prior node in the network shown with the associated variable called as priorKnowledge. This prior node is linked to the Knowledge node, which is a hidden node representing the latent variable LearnRate and needs to be learned during the process of game-play. This node is in turn connected to the Question node which has two states correct or incorrect, depending on whether the player answers the question correctly or incorrectly. Depending on the status of the prior node and question node, learn rate is calculated and is used as a prior for the next level, once the orevious levels gets cleared.

I have the following piece of code which represents the network in C# using the Bayes server library.

BayesServer.Network beliefnet;
Variable priorKnowledge;
Node priorKnowledgeNode;
Variable learnRate;
Node knowledgeNode;
Node questionNode;
ParameterLearning learning;
ParameterLearningOptions learningOptions;

// numberOfDistractors and levelId will be used later for added complexity in modeling
void InitializeNetworkForLevel(int numberOfDistractors, int levelId)
{
    beliefnet = new BayesServer.Network();

    // add an intial knowledge node
    priorKnowledge = new Variable("PriorKnowledge", VariableValueType.Continuous, VariableKind.Probability);
    // initialize the priorKnowledge value to 0.5 if level = 1, else set it to learn rate (may be here, or in the queryNetwork method)
    priorKnowledgeNode = new Node("Prior", priorKnowledge);
    beliefnet.Nodes.Add(priorKnowledgeNode);

    // add a knowledge node which is a latent variable (parameter to be learned from observed values
    learnRate = new Variable("LearnRate", VariableValueType.Continuous, VariableKind.Probability);
    knowledgeNode = new Node("Knowledge", learnRate);
    beliefnet.Nodes.Add(knowledgeNode);

    // add a link from prior node to knowledge node
    beliefnet.Links.Add(new Link(priorKnowledgeNode, knowledgeNode));

    // add a question node, which denotes the oberved variable whether the question is answered correctly or not
    // this node has two states, namely correct or incorrect
    State correct = new State("Correct");
    State inCorrect = new State("Inorrect");
    questionNode = new Node("Question", correct, inCorrect);
    beliefnet.Nodes.Add(questionNode);

    // add a link from knowledge node to question node
    beliefnet.Links.Add(new Link(knowledgeNode, questionNode));

    // We will use the RelevanceTree algorithm here, as it is optimized for parameter learning
    learning = new ParameterLearning(beliefnet, new RelevanceTreeInferenceFactory());
    learningOptions = new ParameterLearningOptions();
}

Also, following is the QueryNetwork() method. However, I am not clear about a lot of things regarding it which are mentioned in comments as well. Firstly, I need to create an EvidenceReaderCommand object for parameter learning which needs 3 parameters: readerCommand to read non temporal data, variableList for the non temporal variables, and ReaderOptions. variableList needs a list of variables which I am not sure what should be in this case. readerCommand needs a table with rows and columns, which again I am not sure what should be in this case. Could anyone please help me in getting a better clarity of these two and how to build them properly for this network?

void QueryNetwork()
{
    DataColumn column;
    DataRow row;

    // don't know which nodes go into this table :(
    // is this supposed to be the conditional probability table? where will the probabilities come from?
    // most probably I'd need to add more columns, but not sure right now
    DataTable table = new DataTable("ParentTable");
    column = new DataColumn();
    column.DataType = System.Type.GetType("System.Double");
    column.ColumnName = "LearnRate";
    column.ReadOnly = true;
    column.Unique = true;
    // i am not sure but table needs to have some rows of data as well, which will be their conditional probabilities
    table.Columns.Add(column);

    IDataReaderCommand readerCommand = new DataTableDataReaderCommand(table);

    // don't know which variables should be a part of this variableList
    IList<VariableReference> variableList = new List<VariableReference>();
    variableList.Add(new VariableReference(learnRate, ColumnValueType.Value, "LearnRate"));

    EvidenceReaderCommand evidenceReaderCommand = new EvidenceReaderCommand(readerCommand, variableList, new ReaderOptions(null));
    ParameterLearningOutput result = learning.Learn(evidenceReaderCommand, learningOptions);
    Debug.Log(result.LogLikelihood.Value);
}

Answer 1

The following code demonstrates how to create a simple Dynamic Bayesian network, similar to the one shown in your image.

var network = new Network();

var knowledge = new Variable("Knowledge", new string[] { "False", "True" });
var nodeKnowledge = new Node(knowledge)
{
    TemporalType = TemporalType.Temporal // this is a time series node, hence re-used for each time slice
};
network.Nodes.Add(nodeKnowledge);

var question = new Variable("Question", new string[] { "Incorrect", "Correct" });
var nodeQuestion = new Node(question)
{
    TemporalType = TemporalType.Temporal  // this is a time series node, hence re-used for each time slice
};
network.Nodes.Add(nodeQuestion);

network.Links.Add(new Link(nodeKnowledge, nodeKnowledge, 1));   // time series link (order/lag 1)
network.Links.Add(new Link(nodeKnowledge, nodeQuestion, 0));

This will create a DBN as shown in the image below. (this is actually just a simple Hidden Markov model, but can be extended to create more sophisticated models)

If this DBN were to be unrolled (not required for inference) it would look like this (in compact view):

In order to learn the parameters of the network, see the following link, which can be modified to learn this network.

Parameter learning code example in C#

To see how to layout you data see the following link:

Data and temporal data layout