3

I am totally new to machine learning, I am currently playing with MNIST machine learning, using RandomForestClassifier.

I use sklearn and panda. I have a training CSV data set.

import pandas as pd
import numpy as np
from sklearn import model_selection
from sklearn.ensemble import RandomForestClassifier
from sklearn.svm import LinearSVC
from sklearn.linear_model import SGDClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score

train = pd.read_csv("train.csv")
features = train.columns[1:]
X = train[features]
y = train['label']

user_train = pd.read_csv("input.csv")
user_features = user_train.columns[1:]
y_train = user_train[user_features]
user_y = user_train['label']


X_train, X_test, y_train, y_test = model_selection.train_test_split(X/255.,y,test_size=1,random_state=0)

clf_rf = RandomForestClassifier()
clf_rf.fit(X_train, y_train)
y_pred_rf = clf_rf.predict(X_test)
acc_rf = accuracy_score(y_test, y_pred_rf)

print("pred : ", y_pred_rf)
print("random forest accuracy: ",acc_rf)

I have the current code, which works well. It takes the training set, split and take one element for testing, and does the prediction.

What I want now is to use the testing data from an input, I have a new csv called "input.csv", and I want to predict the value inside this csv.

How can I replace the model_selection.train_test_split with my input data ? I am sure the response is very obvious, and I didn't find anything.

user2724028
  • 594
  • 2
  • 8
  • 19
  • Do you want to re-train the model using the input.csv data, or just take the fitted model and apply it to input.csv as your test data? – Brad Solomon Jun 01 '17 at 22:36

1 Answers1

3

The following part of your code is unused

user_train = pd.read_csv("input.csv")
user_features = user_train.columns[1:]
y_train = user_train[user_features]
user_y = user_train['label']

If input.csv has the same structure of train.csv you may want to:

  • train a classifier and test it on a split of the input.csv dataset: (please refer to http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html to know how to set the test size)

    input_train = pd.read_csv("input.csv")    
input_features = user_train.columns[1:]    
input_data = user_train[input_features]    
input_labels = user_train['label']
data_train, data_test, labels_train, labels_test = model_selection.train_test_split(input_data/255.,input_labels,test_size=1,random_state=0)

clf_rf = RandomForestClassifier()
clf_rf.fit(data_train, labels_train)
labels_pred_rf = clf_rf.predict(data_test)
acc_rf = accuracy_score(labels_test, labels_pred_rf)

  • test the previously trained classifier on the whole input.csv file

    input_train = pd.read_csv("input.csv")    
input_features = user_train.columns[1:]    
input_data = user_train[input_features]    
input_labels = user_train['label']

labels_pred_rf = clf_rf.predict(input_data)
acc_rf = accuracy_score(input_labels, labels_pred_rf)
ale-6
  • 365
  • 5
  • 14