Iterating a Numpy Array for random.normal generator shape conflict

Question

I am trying to generate n (in this case n=57) random numbers from a normal distribution for a number of sampled mean and standard deviations from a PyMc3 model (in this case 350). So I ultimately want to end up with 350 distributions of 57 length each. I'm sure this is something straightforward and I have a lack of conceptual understanding. Input is:

 prior_pc5 =pm.sample_prior_predictive(samples=350,model=model_5,
    var_names='μ','σ'],random_seed=21)

    n=57

    prpc5_μ = np.asarray(prior_pc5['μ'])
    prpc5_σ = np.asarray(prior_pc5['σ'])

for x,y in np.nditer([prpc5_μ,prpc5_σ]):
    y_prpc5 = np.random.normal(prpc5[:,0],prpc5[:,1], size=n)

Output is:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-180-90195f458d14> in <module>
      1 for x,y in np.nditer([prpc5_μ,prpc5_σ]):
----> 2     y_prpc5 = np.random.normal(prpc5[:,0],prpc5[:,1], size=n)

mtrand.pyx in numpy.random.mtrand.RandomState.normal()

_common.pyx in numpy.random._common.cont()

_common.pyx in numpy.random._common.cont_broadcast_2()

__init__.pxd in numpy.PyArray_MultiIterNew3()

ValueError: shape mismatch: objects cannot be broadcast to a single shape

Appreciate any edification you can provide.

the interwebs encouraged nditer to iterate through an array. prpc5 is defined in the complete code, but I omitted it for brevity. In any case, when I turn the prpc5 dict into individual arrays I have two 1D arrays of length 350. — NavArch, Mar 01 '20 at 15:07

hpaulj · Answer 1 · 2020-03-01T00:51:58.300

Your nditer loop does nothing for you. You don't even use the x,y variables. The prpc5 variable is undefined. And there's no attempt to accumulate the y_prpc5 values.

If you need to iterate on something, start with a plain iteration. Don't attempt to use nditer (unless you can read and understand all of its docs). It's not any faster, and harder to use correctly.

But the error has nothing to do with nditer.

np.random.normal(prpc5[:,0],prpc5[:,1], size=n)

doesn't use any iteration variables.

Size with scalar arguments:

In [63]: np.random.normal(1,2,size=57)                                                         
Out[63]: 
array([-0.15095176,  0.68354153,  0.64270214,  1.71539129,  3.82930345,
       -0.93888021,  0.34013012,  4.7390541 ,  1.95647694, -0.02787572,
        0.53790931,  3.64859278, -2.66455301, -1.81567149,  2.62141742,
       -0.22887411, -0.36284743,  2.92298403,  1.87943503,  2.12060767,
       -1.10172555,  0.04234386,  0.48707306,  5.66358341,  0.70659526,
       -0.74210809, -2.04678512, -0.16496427, -0.46041457,  0.50505178,
        1.66497518,  2.20486689,  1.83034991, -1.73740446, -3.117619  ,
        1.12649528,  2.58059286,  1.42897399,  2.37256695, -2.34670202,
        3.00318398,  2.78164509, -1.1329154 ,  4.06859674,  3.13266299,
       -0.35481326,  1.79429889,  1.71617491,  1.41543611,  0.9476942 ,
       -0.79856396, -0.83121952, -2.63145461,  0.13941223,  0.18895024,
        3.21956521, -2.75348353])

Array/list arguments with size:

In [64]: np.random.normal([1,2],[1,1],size=57)                                                 
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-64-9a493d59b8d2> in <module>
----> 1 np.random.normal([1,2],[1,1],size=57)

mtrand.pyx in numpy.random.mtrand.RandomState.normal()

_common.pyx in numpy.random._common.cont()

_common.pyx in numpy.random._common.cont_broadcast_2()

__init__.pxd in numpy.PyArray_MultiIterNew3()

ValueError: shape mismatch: objects cannot be broadcast to a single shape

If size matches the size of the first two arguments, ok:

In [65]: np.random.normal([1,2],[1,1],size=2)                                                  
Out[65]: array([1.91404732, 2.79305575])

To clarify I would like to generate 350 samples from a normal distribution with each sample 57 in length. How would I do this with np.random.normal? — NavArch, Mar 01 '20 at 15:08

Iterating a Numpy Array for random.normal generator shape conflict

1 Answers1