2

How to convert two datasets X and Y to histograms whose x-axes/index are identical, instead of the x-axis range of variable X being collectively lower or higher than the x-axis range of variable Y (like how the code below generates)? I would like the numpy histogram output values to be ready to plot in a shared histogram-plot afterwards.

import numpy as np
from numpy.random import randn

n = 100  # number of bins

#datasets
X = randn(n)*.1
Y = randn(n)*.2

#empirical distributions
a = np.histogram(X,bins=n)
b = np.histogram(Y,bins=n)
develarist
  • 1,224
  • 1
  • 13
  • 34

1 Answers1

2

You need not use np.histogram if your goal is just plotting the two (or more) together. Matplotlib can do that.

import matplotlib.pyplot as plt

plt.hist([X, Y])  # using your X & Y from question
plt.show()

enter image description here

If you want the probabilities instead of counts in histogram, add weights:

wx = np.ones_like(X) / len(X)
wy = np.ones_like(Y) / len(Y)

You can also get output from plt.hist for some other usage.

n_plt, bins_plt, patches = plt.hist([X, Y], bins=n-1, weights=[wx,wy])  
plt.show()

enter image description here

Note usage of n-1 here instead of n because one extra bin is added by numpy and matplotlib. You may use n depending on your use case.

However, if you really want the bins for some other purpose, np.historgram gives the bins used in output - which you can use as input in second histogram:

a,bins_numpy = np.histogram(Y,bins=n-1)
b,bins2 = np.histogram(X,bins=bins_numpy)

Y's bins used for X here because your Y has wider range than X.

Reconciliation Checks:

all(bins_numpy == bins2)

>>>True


all(bins_numpy == bins_plt)

>>>True
Abhi25t
  • 3,703
  • 3
  • 19
  • 32
  • Thanks, another question, how do i convert the y-axis of your graph from frequencies to probabilities whose max is 1? somehow normalize a and b? – develarist Nov 02 '20 at 16:47
  • 1
    @develarist Just divide the histogram by total number of elements, like this: a/sum(a) – Abhi25t Nov 02 '20 at 17:26
  • I tried to normalize the frequency of the histogram into probabililities using `plt.hist([X, Y], density=True)`, but the histogram produced by `density=True` is identical to `density=False` – develarist Nov 02 '20 at 17:49
  • No, do not do that. When you get numpy histogram a using the command `a, bins1 = np.histogram(X,bins=n)` simply divide the histogram by total number of elements `probability_dist = a/sum(a)` – Abhi25t Nov 02 '20 at 19:39
  • but how do I graph that transformation as a histogram after? the two outputs of `np.histogram` (histogram values, and bin edges) are shaped `len(a)` and `len(a)+1`, giving a shape mismatch error if I try to plot these vectors manually after, transformed or untransormed – develarist Nov 03 '20 at 05:40
  • Oh, now I got your goal. Adding that to the answer – Abhi25t Nov 03 '20 at 06:28
  • in your edit, how do I transform the X and Y frequencies *directly* to probabilities. Do I multiple the original, untransformed histogram values by the normalization weights `wx`? – develarist Nov 03 '20 at 06:37
  • As discussed before in comments `probability_dist = a/sum(a)` – Abhi25t Nov 03 '20 at 11:14
  • but `plt.hist` requires weights and doesn't use the direct transformation `a/sum(a)`. How to reconcile the former, that masks the original data, and the latter, which transforms the data? I need both at the same time, not one or the other – develarist Nov 03 '20 at 11:45
  • Yeah `a,bins1 = np.histogram(X,bins=n)` is giving 101 bins instead of 100. What if we change `n` to `n-1` here ? i.e. `a,bins1 = np.histogram(X,bins=n-1)`. Does that help ? – Abhi25t Nov 03 '20 at 13:24
  • that's not the issue of my previous comment, but regarding `bins=n-1`, how would you know whether to remove the left or right bin edge? – develarist Nov 03 '20 at 13:47
  • `bins=n-1` will give exact 100 bins. No need to manually remove any left/right bin. Anyway I think your main issue is **reconciliation**. Also, I think I was confused because your number of elements in X and number of bins is equal. `X = randn(n)` and `bins=n`. I am assuming that is intentional and editing the answer. – Abhi25t Nov 03 '20 at 14:56