Use AdaBoost(Boosting) with Accord.Net

Question

I am trying to use adaboost (or boosting) in Accord.Net. I tried a version of the example given by https://github.com/accord-net/framework/wiki/Classification for decision trees and it works well with the following code:

'' Creates a matrix from the entire source data table
Dim data As DataTable = CType(DataView.DataSource, DataTable)

'' Create a new codification codebook to 
'' convert strings into integer symbols
Dim codebook As New Codification(data)

'' Translate our training data into integer symbols using our codebook:
Dim symbols As DataTable = codebook.Apply(data)
Dim inputs As Double()() = symbols.ToArray(Of Double)("Outlook", "Temperature", "Humidity", "Wind")
Dim outputs As Integer() = symbols.ToArray(Of Integer)("PlayTennis")

'' Gather information about decision variables
Dim attributes() As DecisionVariable = {New DecisionVariable("Outlook", 3), New DecisionVariable("Temperature", 3), _
    New DecisionVariable("Humidity", 2), New DecisionVariable("Wind", 2)}

Dim classCount As Integer = 2 '' 2 possible output values for playing tennis: yes or no

''Create the decision tree using the attributes and classes
tree = New DecisionTree(attributes, classCount)

'' Create a new instance of the ID3 algorithm
Dim Learning As New C45Learning(tree)

'' Learn the training instances!
Learning.Run(inputs, outputs)

Dim aa As Integer() = codebook.Translate("D1", "Rain", "Mild", "High", "Weak")

Dim ans As Integer = tree.Compute(aa)

Dim answer As String = codebook.Translate("PlayTennis", ans)

Now I want to addapt this code to use adaboost or boosting on more complicated examples. I tried the following by adding the following to the code above:

Dim Booster As New Boost(Of DecisionStump)()

Dim Learn As New AdaBoost(Of DecisionStump)(Booster)
Dim weights(inputs.Length - 1) As Double
For i As Integer = 0 To weights.Length - 1
    weights(i) = 1.0 / weights.Length
Next

Learn.Creation = New ModelConstructor(Of DecisionStump)(x=>tree.Compute(x))
Dim Err As Double = Learn.Run(inputs, outputs, weights)

The problem seem to be the line:

Learn.Creation = New ModelConstructor(Of DecisionStump)(x=>tree.Compute(x))

How can I use adaboost or boosting in Accord.Net? How can I adjust my code to make it work? All help will be appreciated.

Did you define "x"?, the algorithm might not like an undefined variable if it's not defined. Looking at the specific Ying-Yang dataset, you could use model-based clustering (Gaussian mixture models) and probably do better than any of the supervised methods shown. Also, an SVM with a radial basis function (RBF) kernel should do well. (I did not look at the SVM methodology used). — , Jan 13 '17 at 22:57

score 1 · Answer 1 · edited Jan 20 '21 at 08:16

This is a late response, but to those who might find it useful in the future, since release 3.8.0, a Boosted Decision Tree can be learned using the Accord.NET Framework as shown below:

// This example shows how to use AdaBoost to train more complex
// models than a simple DecisionStump. For example, we will use
// it to train a boosted Decision Trees.

// Let's use some synthetic data for that: The Yin-Yang dataset is 
// a simple 2D binary non-linear decision problem where the points 
// belong to each of the classes interwine in a Yin-Yang shape:
var dataset = new YinYang();
double[][] inputs = dataset.Instances;
int[] outputs = Classes.ToZeroOne(dataset.ClassLabels);

// Create an AdaBoost for Logistic Regression as:
var teacher = new AdaBoost<DecisionTree>()
{
    // Here we can specify how each regression should be learned:
    Learner = (param) => new C45Learning()
    {
        // i.e.
        // MaxHeight = 
        // MaxVariables = 
    },

    // Train until:
    MaxIterations = 50,
    Tolerance = 1e-5,
};

// Now, we can use the Learn method to learn a boosted classifier
Boost<DecisionTree> classifier = teacher.Learn(inputs, outputs);

// And we can test its performance using (error should be 0.11):
double error = ConfusionMatrix.Estimate(classifier, inputs, outputs).Error;

// And compute a decision for a single data point using:
bool y = classifier.Decide(inputs[0]); // result should false

Use AdaBoost(Boosting) with Accord.Net

1 Answers1