0

My Problem is to get better result in MSE and i hope in R2 as in linear model. So i used the sklearn.neural_network.MLPRegressor library to compare it

def testfit(v,k,subset=2,hls=(50,50,50,10),acv='identity'): 
# prep variables
n = len(v)
n1 = n/subset
X = getX(v[0:n1],k)
y = gety(v[0:n1],k)
# define models
nn1 = MLPRegressor(hidden_layer_sizes=hls,  activation=acv, solver='adam',    alpha=0.01,batch_size='auto',
       learning_rate='constant', learning_rate_init=0.1, power_t=1, max_iter=50000, shuffle=True,
       random_state=None, tol=0.00001, verbose=False, warm_start=False, momentum=0.9,
       nesterovs_momentum=True, early_stopping=False, validation_fraction=0.5, beta_1=0.9, beta_2=0.999,
       epsilon=1e-10)


ols = linear_model.LinearRegression()
# run models
st = time.time()
fnnw = nn1.fit(X,y)
nnw_dur = time.time() - st
st = time.time()
flin = ols.fit(X,y)
ols_dur = time.time() - st
# run gof   
X2 = getX(v[n1:n],k)
y2 = gety(v[n1:n],k)
# neural network
# in-sample
yn = fnnw.predict(X)
gin = pearsonr(y,yn)[0]**2
ginse = sum((y-yn)**2)
# out-sample    
yn2 = fnnw.predict(X2)
oin = pearsonr(y2,yn2)[0]**2
oinse = sum((y2-yn2)**2)
# ols
# in.sample
yl = flin.predict(X)
gil = pearsonr(y,yl)[0]**2
gilse = sum((y-yl)**2)
yl2 = flin.predict(X2)
oil = pearsonr(y2,yl2)[0]**2
oilse = sum((y2-yl2)**2)
plt.subplot(321)
plt.plot(y2)
plt.plot(yl2)

enter image description here

The best case in this scenario is that my neural network NNW MSE in FORCAST +1 is smaller than OLS MSE FORCAST +1

Or is it not possible to get a smaller Error in nn as in linear model in this way

Moritz
  • 47
  • 6
  • Looking at your activation function, you're using the `identity` function, which means it will perform no better than linear regression. – Scratch'N'Purr Mar 02 '18 at 13:46
  • thanks, i used relu tanh logistic but the result is not better....and another consideration is it useful to use tensorflow to get better result in mse – Moritz Mar 02 '18 at 13:53
  • Are your inputs scaled between 0 and 1 or -1 and 1? Also, looking at your hidden layers, 4 might be overkill. In most cases, 1-2 hidden layers are good enough. And I would keep the number of neurons between 1 to the number of columns. – Scratch'N'Purr Mar 02 '18 at 14:22
  • inputs scaled between 0 and 1 .....columns are 10 – Moritz Mar 02 '18 at 14:28
  • I changed : hls=2 , acv='logistic' and my results are not much better – Moritz Mar 02 '18 at 15:50
  • Did you try `hls=(9, 3)`? This denotes 2 hidden layers, where the 1st layer has 9 neurons (which is less than 10 columns), and the 2nd layer with 3 neurons. You could try `reLU` as well. Also, another issue could be that your NN could be overfitting with your training data, though it doesn't look the case from your image. Ultimately, its really about experimentation with different models and hyperparameter settings. You could probably even apply a grid search on your NN to find the optimal layers/neurons. – Scratch'N'Purr Mar 02 '18 at 16:40

0 Answers0