According to MATLAB documentation,
graycomatrix calculates the GLCM from a scaled version of the image.
By default, if I is a binary image, graycomatrix scales the image to
two gray-levels. If I is an intensity image, graycomatrix scales the
image to eight gray-levels. You can specify the number of gray-levels
graycomatrix uses to scale the image by using the 'NumLevels'
parameter, and the way that graycomatrix scales the values using the
'GrayLimits'
parameter — see Parameters.
In short, you need to run the function as follows:
glcm = graycomatrix(I , 'NumLevels' , 2^8 );