Stack Overflow
Stack Overflow

Questions tagged [hdf]

Hierarchical Data Format (HDF, HDF4, or HDF5) is a set of file formats and libraries designed to store and organize large amounts of numerical data.

Hierarchical Data Format (HDF, HDF4, or HDF5) is a set of file formats and libraries designed to store and organize large amounts of numerical data.

Originally developed at the National Center for Supercomputing Applications, it is supported by the non-profit HDF Group, whose mission is to ensure continued development of HDF5 technologies, and the continued accessibility of data stored in HDF.

In keeping with this goal, the HDF format, libraries and associated tools are available under a liberal, BSD-like license for general use. HDF is supported by many commercial and non-commercial software platforms, including Java, MATLAB/Scilab, Octave, IDL, Python, and R. The freely available HDF distribution consists of the library, command-line utilities, test suite source, Java interface, and the Java-based HDF Viewer (HDFView).

There are two major versions of HDF; HDF4 and HDF5, which differ significantly in design and API.

Wikipedia: http://en.wikipedia.org/wiki/Hierarchical_Data_Format

344 questions
1
vote
1 answer

Reading and working with FireModis HDF files in R

I`m trying to read a Modis Fire HDF file using the instructions provided by the hdfeos information center (https://hdfeos.org/software/r.php), this is the…
Jaime
  • 111
  • 7
1
vote
1 answer

Nifi PutHiveStreaming writing data to HDFS but not able to see data in Hive table

I'm using HDF Nifi 1.5 and Hdp Hive 3.1. Create a partioned and bucketing table in Hive: CREATE TABLE default.client_tbl ( client_key int, clt_name varchar(45), clt_description varchar(200), version int ) partitioned by (clt_status varchar(8))…
Karthik Mannava
  • 207
  • 1
  • 5
  • 12
1
vote
1 answer

Convert HDF4 to raster; with longitude/latitude grids are available in another HDF file

I am trying to convent HDF4 files (representing daily Sea Ice Concentration) to raster object in R. However, the HDF files themselves do not contain the longitude/latitude grids or projection information, and such information should be extracted…
Ahmed El-Gabbas
  • 398
  • 3
  • 10
1
vote
0 answers

Defining dtypes at time of import of a tab-delimited file into a dataframe

As some data are ambiguous (e.g customer numbers that should be interpreted as strings and not integers), I am using the dtype option (pd.read_table('BSC.csv', dtype=str). It works fine,as Pandas do not complain anymore about ambiguous…
JCF
  • 307
  • 2
  • 17
1
vote
2 answers

How to Install PyHDF package on Google Colab?

I tried !pip install pyhdf, Referred this too. http://stream.princeton.edu/AWCM/LIBRARIES/pyhdf-0.8.3/doc/webpage/source/install.html no luck. Kindly help.
Savio Kay
  • 11
  • 1
  • 4
1
vote
0 answers

OpenCV 3.3.1 and HDF5 Image Read

I'm trying to read from an HDF5 dataset file. The file is a .h5 dataset created in python (possibly h5py) and contains 2 attributes. The first attribute is a depth image - "/depth" (single channel, 32bit float matrix) with dimensions 376 x 1241. The…
Shreyas Skandan
  • 11
  • 4
1
vote
3 answers

lat,lon information from hdf file python

I have a hdf file and want to extract data from it. For some reason i cant able to extract latitude and longitude values: the code that i tried is : from pyhdf import SD hdf = SD.SD('MOD10C2.A2001033.006.2016092173057.hdf') data =…
bikuser
  • 2,013
  • 4
  • 33
  • 57
1
vote
1 answer

HDFql working with tables

I have some question on the usability of HDFql: Does HDFql support created of a table where the row are of different type? How do you append data to the table? How do you iterate though the row do the table? The table I want to create will have…
Johan
  • 575
  • 5
  • 21
1
vote
1 answer

H5dread_f symbol missing in hdf5 (fortran) libraries

I am facing an issue while compiling WRF-DA code (code is here) The compilation line which fails - ftn -c -ip -O3 -w -ftz -fno-alias -align all -FR -convert big_endian -r8 -real-size `expr 8 \* 8` -i4 -I../external/crtm_2.2.3/libsrc …
puneet336
  • 433
  • 5
  • 20
1
vote
1 answer

Viewing a large (12.5GB) HDF5 file written using h5py on HDFView 2.14

I have a very large HDF5 file created using Python/ h5py. I cannot open the file on HDFView 2.14, when I try to open the file nothing happens. Any suggestions on how I can open/ view the file? It contains just 5 datasets, but each dataset has 778…
Mridula Gunturi
  • 157
  • 8
1
vote
1 answer

Unable to read HDF4 dataset using pyhdf (pyhdf.error.HDF4Error: select: non-existent dataset )

I am trying to read a HDF4 file (https://www.dropbox.com/s/5d40ukfsu0yupwl/MOD13A2.A2016001.h23v05.006.2016029070140.hdf?dl=0). import os import numpy as np from pyhdf.SD import SD, SDC # Open file. FILE_NAME =…
Yogesh Sathe
  • 41
  • 1
  • 7
1
vote
0 answers

Appending to a file without closing it in HDFS

I am trying to append some text in HDFS, but it turns out that I have to close the output stream fin_a.close() after every appending. Is there any way which I can append to a file without closing it? try{ Path filenamePath = new Path(appId +…
Self
  • 147
  • 2
  • 16
1
vote
1 answer

Converting HDF to georeferenced file (geotiff, shapefile)

I am dealing with 28 HDF4 files on ocean primary productivity (annual .tar files can be found here: http://orca.science.oregonstate.edu/1080.by.2160.monthly.hdf.cbpm2.v.php) My goal is to do some calculations (I need to calculate concentrations per…
Kristina
  • 263
  • 2
  • 5
  • 11
1
vote
0 answers

How many string type fields can be written in a compound dataset in HDF5 file?

I am using Hdf5DotnetTools, a C# wrapper on HDF.Pinvoke version 1.10. in Windows 10 and HDFViewer version 3.0. I ran the test cases in Debug mode and found that only one string type field can be written in a compound dataset. inside the HDFviewer…
Shubhjot
  • 73
  • 1
  • 8
1
vote
1 answer

How to read delta records from MS SQL using Nifi/HDF

I have a few tables in MS SQL, these tables gets updated every second and query more or less look like this SELECT G_ID,UpdateTime,ID,Name,T_NAME FROM TABLE1 AS table1 INNER JOIN TABLE2 AS table2 ON table1.IP=table2.ID WHERE table2.UpdateTime >=…
nilesh1212
  • 1,561
  • 2
  • 26
  • 60
1 2 3
…
22 23