DQN(Reinforcement learning) : should state be standardized?

Question

This is my state dataframe:

>> state_df.head()
          A           B          C
 0    -1.469587   -1.186974   -1.136587
 1    -1.310300   -1.032667   -1.389515
 2    -0.041564   -0.112118   -0.742551
 3     0.698519    0.453808   -0.194451
 4     0.653907    0.425225   -0.157008

Each column is kind of index(in finance) getting from my data set. I'm gonna combine each row and set it as a state like this:

for i in len(state_df):
    state_list = np.array(indicators_df.ix[i].tolist())
    x = np.reshape(state, [-1, input_size])
    session.run(self._Qpred, feed_dict={self._X: x})
    .
    .
    .
    .

And these state gonna be input of DQN(Deep Q-Network).

But each column doesn't follow normal distribution. Their mean() and std() like this :

state_df['A'].mean() => 1.0023571097367265
state_df['A'].std() => 0.039181434958815514
state_df['B'].mean() => 0.08110446799218411
state_df['B'].std() => 0.643645664287425
state_df['C'].mean() => 0.006230702891531177
state_df['C'].std() => 0.06876011348732677

I wonder that I must standardize each column ( (x - mu) / sigma)...

Do I have to?

score 0 · Answer 1 · answered May 11 '17 at 12:28

0

No, it's not necessary.

For example DQN can be used on raw images (ok, probably you'd be grey-scaling them for sake of a reasonable size), grabbed from the video games, where each particular pixel doesn't necessarily follow a standard normal distribution. And it works regardless.

answered May 11 '17 at 12:28

Massyanya

2,844
8
28
37

Then, Do I need not to worry about feature biasing? For example, `x1` feature range 1 ~ 100000, feature `x2` range 1 ~ 10, which could occur an feature biasing? – user3595632 May 12 '17 at 06:25

DQN(Reinforcement learning) : should state be standardized?

1 Answers1