How can I partially read 2d satellite file in Python? (binary, fromfile)

Question

I have a lot of satellite data that is consists of two-dimension.

(I convert H5 to 2d array data that not include latitude information I made Lat/Lon information data additionally.)

I know real Lat/Lon coordination and grid coordination in one data.

How can I partially read 2d satellite file in Python?

"numpy.fromfile" was usually used to read binary file.

if I use option as count in numpy.fromfile, I can read binary file partially.

However I want to skip front records in one data for save memory.

for example, i have 3x3 2d data as follow:

python

a= [[1,2,3] [4,5,6] [7,8,9]]

I just read a[3][0] in Python. (result = 7)

When I read file in Fortran, I used "recl, rec".

Fortran

open(1, file='exsmaple.bin', access='direct', recl=4) ! recl=4 means 4 btype

read(1, rec=lat*x-lon) filename

close(1)

lat means position of latitude in data. (lat = 3 in above exsample ; start number is 1 in Fortran.)

lon means position of longitude in data. (lon = 1 in above exsample ; start number is 1 in Fortran.)

x is no. rows. (x = 3, above example, array is 3x3)

I can read file, and use only 4 byte of memory.

I want to know similar method in Python.

Please give me special information to save time and memory.

Thank you for reading my question.

2016.10.28. Solution

python Data = [1,2,3,4,5,6,7,8,9], dtype = int8, filename=name

a = np.memmap(name, dtype='int8', mode='r', shape=(1), offset=6)

print a[0]

result : 7

Answer 1

To read .h5 files :

import h5py
ds = h5py.File(filename, "r")
variable = ds['variable_name']

Answer 2

It's hard to follow your description. Some proper code indentation would help over come your English language problems.

So you have data on a H5 file. The simplest approach is to h5py to load it into a Python/numpy session, and select the necessary data from those arrays.

But it sounds as though you have written a portion of this data to a 'plain' binary file. It might help to know how you did it. Also in what way is this 2d?

np.fromfile reads a file as though it was 1d. Can you read this file, up to some count? And with a correct dtype?

np.fromfile accepts an open file. So I think you can open the file, use seek to skip forward, and then read count items from there. But I haven't tested that idea.