-1

I'm triyng to develop the system: enter image description here

The architecture is taken from:https://www.researchgate.net/publication/307852688_Neural_Network-Based_Self-Tuning_PID_Control_for_Underwater_Vehicles

So the idea is that NN predicts 3 coefficients for the next step which will provide the minimization of the system error (setpoint minus system output).

I have the model of the system in python app and can imitate the output of the system for each set of coefficients.

But how can I train my NN? As I understand I cannot use supervised learning because I don't have the "correct" coefficients for each set of input states.

Is it a task for reinforcement learning? Or it should be classified as optimization problem?

I've been triyng to build the NN with keras. But the problem is with loss function. It is assumed to receive y_true, y_pred (even I build custom loss func). But I only get model outputs on each step (3 coeffs) and setpoint to compare the system output. Is it possible to build such a outer loss function? Here my attempts:

from tensorflow import keras
from tensorflow.keras import layers

inputs = keras.Input(shape=(1,4))
x = layers.Dense(4, activation='sigmoid', name='dense_1')(inputs)
x = layers.Dense(12, activation='sigmoid', name='dense_2')(x)
outputs = layers.Dense(3, activation='linear', name='predictions')(x)

model = keras.Model(inputs=inputs, outputs=outputs)  

SP=sim_output.SetPoint

def s_e(SP,model_outputs):
                   
        Kp,Ki,Kd=model_outputs
        P=tf(Kp,1)
        I=tf(Ki,[1,0])
        D=tf([Kd,0],[0,1])
        controller=P+I+D
        system1=controller*drive
        system=feedback(system1,Ks)
    
        t_step=[0]
        y_step=[0]
        #print('SP_next=0.44, pv=0')
        T_step=T[0:462]
        custom_step=custom[0:462]    
        t_step,y_step=control.input_output_response(io_sys,T_step,custom_step)
        
        SP=custom_step[0]
        yv=y_step[len(y_step)-1]
        
        s_e=SP-yv
        return s_e

model.compile(optimizer='adam', loss=s_e(SP,outputs), metrics=['mae'])

Got:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-30-6aa3b59fe166> in <module>
      1 model.compile(optimizer='adam', 
----> 2               loss=s_e(SP,outputs),
      3               metrics=['mae'])

<ipython-input-29-119173321a4f> in s_e(SP, model_outputs)
      3 def s_e(SP,model_outputs):
      4 
----> 5         Kp,Ki,Kd=model_outputs
      6         P=tf(Kp,1)
      7         I=tf(Ki,[1,0])

~\Anaconda3\lib\site-packages\tensorflow_core\python\framework\ops.py in __iter__(self)
    554     if shape[0] is None:
    555       raise TypeError(
--> 556           "Cannot iterate over a tensor with unknown first dimension.")
    557     for i in xrange(shape[0]):
    558       yield self[i]

TypeError: Cannot iterate over a tensor with unknown first dimension.

Discovering those question wasn't a big help: https://stats.stackexchange.com/questions/310308/can-neural-network-with-unsupervised-learning-minimize-a-cost-function , How does it work Q-learning+NN

Does anyone have ideas for building such a NN? Thanks in advance!

Victoria
  • 395
  • 3
  • 13

1 Answers1

0

After a great deal of research I'd like to share the solution I've found.

The state of art of the issue is the following:

  1. The common way to train Neural network(NN) for fine-tuning of PID controller is 'Supervisory learning' so just NN learns from the coefficients array, which was gotten by manual tuning and accepted as the best tunes.
  2. If we want to have NN which is superior to human (it was actually my goal) than we really should use reinforcement leaning. But until now this problem wasn't solved completly, and we have a very few papers on it.
  3. Optimization problem can be solved by means of NN, but it is pretty uncommon way. Other algorithms work for this problem better.

So, for now I content with supervisory learning (I picked the training data from Simulink/Matlab model). And may be inthe future i will discover reinforcement learning for this issue.

Victoria
  • 395
  • 3
  • 13