0

I'm currently using tensorflow's built in DNN classifier (tf.estimator.DNNClassifier). Got the model to run successfully however when I view the summary scalars on tensorboard, it literally has everything I need except for training accuracy.

I searched all over but only found solutions to custom built estimators. I was wondering if anyone here was able to get the training accuracy using tf.estimator.DNNClassifier.

Any tips or suggestions are welcomed. Thanks in advance.

MajesticKhan
  • 158
  • 1
  • 11

1 Answers1

0

If you have a trained DNNClassifier then you can use a a built in method of the same class to get the accuracy. You can use the evaluate() method. Just make sure you define your input function according to you evaluation/testing data. Here's an example of fitting a model on random data, evaluating the model on more random data and then printing the summary of the evaluation metrics, including the accuracy:

import numpy as np
import tensorflow as tf
import pandas as pd

num = 100000

feat1 = np.random.rand(num,1)
feat2 = np.random.rand(num,1)
label = (np.round(np.random.rand(num,1),0))
Data = pd.DataFrame(np.concatenate((feat1,feat2,label),axis=1), columns=['feat1','feat2','label'])
feature_columns = [
    tf.feature_column.numeric_column(key="feat1"),
    tf.feature_column.numeric_column(key="feat2"),
]

model = tf.estimator.DNNClassifier(feature_columns=feature_columns,hidden_units=[8,8,8,8,8,8,8,8,4,2])

input_train = tf.estimator.inputs.pandas_input_fn(
    x=Data,
    y=Data['label'],
    shuffle=True,
    num_epochs = 1
)

model.train(input_fn=input_train,steps=10000)

feat1T = np.random.rand(num,1)
feat2T = np.random.rand(num,1)
labelT = (np.round(np.random.rand(num,1),0))
DataT = pd.DataFrame(np.concatenate((feat1T,feat2T,labelT),axis=1), columns=['feat1','feat2','label'])

input_eval = tf.estimator.inputs.pandas_input_fn(
    x=DataT,
    y=DataT['label'],
    shuffle=True,
    num_epochs = 1
)

metrics = model.evaluate(input_fn=input_eval, steps=100)

print(metrics)

For my randomly random seed, this printed:

{'accuracy': 0.49734375, 'accuracy_baseline': 0.5026562, 'auc': 0.5, 'auc_precision_recall': 0.7486719, 'average_loss': 0.6931635, 'label/mean': 0.49734375, 'loss': 88.72493, 'precision': 0.49734375, 'prediction/mean': 0.5012483, 'recall': 1.0, 'global_step': 782}

Manny
  • 131
  • 3
  • The accuracy you are referring to is testing accuracy. I am looking for training accuracy. – MajesticKhan Oct 28 '18 at 03:39
  • Oops. Sorry mate. I just completely misread that. Can you clarify: are you looking for the accuracy WHILE training, or after the training has completed? A simple way to get training accuracy is just to use the `evaluate()` method but pass the same `input_fn` that you used for training. – Manny Oct 28 '18 at 04:21
  • Sorry for the late response....The issue with the solution that you provided is that the evaluation of the testing data won't occur. I'm trying to view the training accuracy along with the evaluation accuracy to prevent over-fitting – MajesticKhan Oct 30 '18 at 13:26