Multi-input single output regression using Scikit neural networks MLPRegressor

Question

I have a 2D-array data as follows (with M & C as independent variables):

Data enter image description here

I'm trying to model this regression (f(M,C) = y) using the Scikit MLPRegressor. Not knowing how to go about modeling multivariable input, I tried modeling it as two independent single-input problems. How do I use this output to predict the curve for C=2.3, for example? If anyone can suggest a more elegant way to program this problem, I'd be thankful.

Code

# importing module
import numpy as np
import pandas as pd
from pandas import *
from matplotlib import pyplot as plt

from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPRegressor

# reading CSV file
#data = read_csv("data\Ex2.csv")
data = pd.DataFrame([[1,1,1],[2,4.28,5.65],[3,10.04,15.59],[4,18.38,32],[5,29.36,55.90],[6,43.06,88.18],[7,59.52,129.64],[8,78.79,181.02],[9,100.90,243],[10,125.89,316.22]], columns=['M','C=2.1','C=2.5'])

# converting column data to list
M = data['M'].values
C1 = data['C=2.1'].values
C2 = data['C=2.5'].values
C = np.column_stack((C1,C2))

M = M.reshape(M.shape[0],-1)

for i in range(0, 2):

    Xtrain, Xtest, Ytrain, Ytest = train_test_split(M, C[:,i], test_size=0.2, random_state=42)            
    mlp = MLPRegressor(random_state=42, activation='relu', hidden_layer_sizes=(100,100,100,100,100,100,100,100,8))    
    
    mlp.fit(Xtrain, Ytrain.ravel())    
    Yguess = mlp.predict(Xtest)
    Y = mlp.predict(M)
    
    #plt.plot(Xtest[:,0], Ytest, '.')
    #plt.plot(Xtest[:,0], Yguess, 'r.')
    plt.plot(M, C[:,i], 'o')
    plt.plot(M, Y, 'r')

Result enter image description here

Where is your `data = read_csv("data\Ex2.csv")`? Adding your input data — I'mahdi, Mar 23 '22 at 15:56
to be fair, this is a pretty good start so far. The question is how do you wish to compute the values for `c=2.3`. do you intend for it to be some kind of weighted average between `c=2.1` and `c=2.5` and if so what are the weightings ? — D.L, Mar 23 '22 at 16:04
the contents of the file Ex2.csv is given at the beginning of the question. Its the 2D table with M & C as variables — Ronnie India, Mar 23 '22 at 16:07
@RonnieIndia please add this `DataFrame` then we can run your code — I'mahdi, Mar 23 '22 at 16:12
@l'mahdi data = pd.DataFrame([[1,1,1],[2,4.28,5.65],[3,10.04,15.59],[4,18.38,32],[5,29.36,55.90],[6,43.06,88.18],[7,59.52,129.64],[8,78.79,181.02],[9,100.90,243],[10,125.89,316.22]], columns=['M','C=2.1','C=2.5']) — Ronnie India, Mar 23 '22 at 16:46
@RonnieIndia, are you asking how to use a particular model or are you asking the general question of what model to use ? — D.L, Mar 23 '22 at 16:57
@D.L The problem I'm trying to solve is something like this: For C=2.1 and for C=2.5, the variation of the output (y) vs M is known. How do I use this data to predict the output vs M for C=2.3 or for C=3.0 or so on...I got stuck halfway through the solution. Not sure how to proceed from hereon — Ronnie India, Mar 23 '22 at 17:10
@l'mahdi I'd prefer to solve it using scikit-learn. I'd move to other packages only as a last resort. — Ronnie India, Mar 23 '22 at 17:13
is there a specific model in `scikit-learn` that you are trying to implement ? — D.L, Mar 23 '22 at 17:17
@D.L ...not sure about the question. I'm using the neural network function MLPRegressor in the code. — Ronnie India, Mar 23 '22 at 17:19
okay, so you will be most likely implementing `multi layer perception`: https://scikit-learn.org/stable/auto_examples/inspection/plot_partial_dependence.html#sphx-glr-auto-examples-inspection-plot-partial-dependence-py — D.L, Mar 23 '22 at 17:26
I’m voting to close this question because it is not about programming as defined in the [help] but about DL theory and/or methodology - please see the NOTE in https://stackoverflow.com/tags/deep-learning/info — desertnaut, Mar 24 '22 at 10:36

score 1 · Answer 1 · answered Mar 23 '22 at 18:41

IIUC, You need Recurrent Neural Network(RNN) or Long short-term memory(LSTM), you can solve your problem with tensorflow like below:

import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np
import pandas as pd


data = pd.DataFrame([[1,1,1],[2,4.28,5.65],
                     [3,10.04,15.59],[4,18.38,32],
                     [5,29.36,55.90],[6,43.06,88.18],
                     [7,59.52,129.64],[8,78.79,181.02],
                     [9,100.90,243],[10,125.89,316.22]], 
                    columns=['M','C=2.1','C=2.5']) 
arr = data.values[:, 1:]
arr = np.insert(arr, 0, [2.1 , 2.5], axis=0).T
arr = np.lib.stride_tricks.sliding_window_view(arr, 3, axis=1)

X_train = arr[...,:-1]
y_train = arr[..., -1]
X_train = X_train.reshape([-1,2])
y_train = y_train.ravel()

model = tf.keras.models.Sequential()
model.add(tf.keras.layers.SimpleRNN(200, input_shape=(2,1), activation='tanh'))
model.add(tf.keras.layers.Dense(1))
model.compile(loss='mean_absolute_error', optimizer='adam')

model.fit(X_train, y_train, epochs=1000, batch_size=16)

X_test = np.array([[2.3,1]], dtype=np.float64)
res = []
for _ in range(10):
    y_pred = model.predict(X_test)
    X_test = np.append(X_test, y_pred[0][0])[1:][None,...]
    res.append(y_pred[0][0])

# predict res for 2.3 => [4.725103, 10.564905, 19.61012, 30.361404, 43.3811, 60.815994, 86.90681, 118.34455, 154.1196, 181.1613]


M = data['M'].values
C1 = data['C=2.1'].values
C2 = data['C=2.5'].values
C = np.column_stack((C1,C2))
M = M.reshape(M.shape[0],-1)

for i in range(0, 2):
    plt.plot(M, C[:,i])
plt.plot(res)
plt.show()

Output:

...
Epoch 999/1000
2/2 [==============================] - 0s 8ms/step - loss: 13.2078
Epoch 1000/1000
2/2 [==============================] - 0s 10ms/step - loss: 13.0156

@l'mahdi Thank you! This looks similar to what I'm targeting at. Let me try to understand the code as some of the parts are new to me and get back. — Ronnie India, Mar 24 '22 at 08:42
@RonnieIndia, welcome. if this is what you want, please edit your question and say you accept other approach or method and no problem if answer will be without scikit-learn. — I'mahdi, Mar 24 '22 at 13:48
@RonnieIndia, please read [this](https://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work) — I'mahdi, Mar 24 '22 at 13:48
@l'mahdi There seems to be a discrepancy in the logic used. Nevertheless, your solution was useful. So I marked it up. I was able to solve the problem using scikit-learn which I'll post soon — Ronnie India, Mar 24 '22 at 15:01

Ronnie India · Accepted Answer · 2022-03-24T15:22:16.690

I was able to solve the problem as below. Thank you @l'mahdi for your help and to @D.L for your comments

Code solution

# importing module
import numpy as np
import pandas as pd
from pandas import *
from matplotlib import pyplot as plt

from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPRegressor

# reading CSV file
data = pd.DataFrame([[1,1,1],[2,4.28,5.65],[3,10.04,15.59],[4,18.38,32],[5,29.36,55.90],[6,43.06,88.18],[7,59.52,129.64],[8,78.79,181.02],[9,100.90,243],[10,125.89,316.22]], columns=['M','C=2.1','C=2.5'])

# converting column data to array
M = data['M'].values
C1 = data['C=2.1'].values
C2 = data['C=2.5'].values

C = np.concatenate((C1,C2))
I = np.full_like(M,1)
MC1 = np.column_stack((M,I*2.1))
MC2 = np.column_stack((M,I*2.5))
MC3 = np.column_stack((M,I*2.4))
MC = np.concatenate((MC1,MC2))

MC = MC.reshape(MC.shape[0],-1)

mlp = MLPRegressor(random_state=0, activation='relu', hidden_layer_sizes=(100,100,100,100,100,100,100,100,100,100,100,100,8))    

mlp.fit(MC,C)  
Y = mlp.predict(MC3)
    
plt.plot(M, C1)
plt.plot(M, C2)
plt.plot(M, Y, 'r')

Result

Multi-input single output regression using Scikit neural networks MLPRegressor

2 Answers2