3
import numpy as np
np.random.seed(1)
import random
random.seed(2)
import tensorflow as tf
tf.compat.v1.set_random_seed(3)  # graph-level seed
if tf.__version__[0] == '2':
    tf.random.set_seed(4)  # global seed
else:
    tf.set_random_seed(4)  # global seed

from tensorflow.keras.initializers import glorot_uniform as GlorotUniform
from tensorflow.keras import backend as K

init = GlorotUniform(seed=5)(shape=(4, 4))
print(K.eval(init))

[[-0.75889236  0.5744677   0.82025963 -0.26889956]
 [ 0.0180248  -0.24747121 -0.0666492   0.23440498]
 [ 0.61886185  0.05548459  0.39713246  0.126324  ]
 [ 0.6639387  -0.58397514  0.39671892  0.67872125]]  # TF 2

[[ 0.2515846  -0.41902617 -0.7859829   0.41573995]
 [ 0.8099498  -0.6861247  -0.46198446 -0.7579694 ]
 [ 0.29976922  0.0310365   0.5031274   0.314076  ]
 [-0.62062943 -0.01889879  0.7725797  -0.65635633]]  # TF 1

Why the difference? This is creating severe reproducibility problems between the two versions - and this or something else, within the same version's (TF2) Graph vs. Eager. More importantly, can TF1's RNG sequence be used in TF2?

Innat
  • 16,113
  • 6
  • 53
  • 101
OverLordGoldDragon
  • 1
  • 9
  • 53
  • 101

1 Answers1

2

With enough digging - yes. TL;DR:

  • TF2 behavior in TF1: from tensorflow.python.keras.initializers import GlorotUniformV2 as GlorotUniform
  • TF1 behavior in TF2: from tensorflow.python.keras.initializers import GlorotUniform

TF2 essentially executes the first bullet under the hood; GlorotUniform is actually GlorotUniformV2.

Some details:

Found docs - but code itself terminates at some pywrapped compiled code (TF1 -- TF2 -- for some reason Github refuses to show gen_stateless_random_ops for TF2 and gen_random_ops for TF1, but you can find both in the local install):

tensorflow.python.ops.gen_random_ops.truncated_normal Outputs random values from a truncated normal distribution.

The generated values follow a normal distribution with mean 0 and standard deviation 1, except that values whose magnitude is more than 2 standard deviations from the mean are dropped and re-picked.

tensorflow.python.ops.gen_stateless_random_ops.truncated_normal Outputs deterministic pseudorandom values from a truncated normal distribution.

The generated values follow a normal distribution with mean 0 and standard deviation 1, except that values whose magnitude is more than 2 standard deviations from the mean are dropped and re-picked.

The outputs are a deterministic function of shape and seed.

The first and second are ultimately where GlorotUniform and GlorotUniformV2 route to, respectively. TF2's from tensorflow.keras.initializers imports from init_ops_v2 (i.e. V2), whereas TF1's from init_ops.

OverLordGoldDragon
  • 1
  • 9
  • 53
  • 101