1

I try ILNumerics.IO.HDF5 and can not read the following data:

  • Variable length strings in Datasets and Attributes.
  • Datasets with variable length arrays. Each cell contain a array of numbers, which are histograms.
  • Compound data, ie. Datasets with structs containing some numbers.

In HDFView 2.10.1 I can read this data: https://anonfiles.com/file/13756916026cafc4e4ec7c333f235bda

How can I use ILNumerics.IO.HDF5 with this data?

I found an other post with suggestion to read string as char. But with the variable length string an exception is thrown: "Error reading data from the attribute!"

 var file = new H5File("test.h5");
 H5Dataset ds1 = file.First<H5Dataset>("Wind");
 var att = ds1.Attributes["Aggregator"];
 var value = att.Get<char>();
Pete
  • 673
  • 5
  • 8

1 Answers1

2

Could you provide more info on how you write the string attributes and what exactly is the issue. When you say 'can not read',Do you get a null return value or do you get an exception.

I write strings as attributes in my application and it works fine. I am guessing there could be a problem in the way you write the string. As per Haymo's suggestion, I convert the string into char array and write as attribute. Here is the sample code

 private ILRetArray<Char> ConvertStringToArray(string str)
        {
            using (ILScope.Enter())
            {
                ILArray<Char> A = ILMath.array<Char>(' ', 1, str.Length);
                for (int i = 0; i < str.Length; i++)
                {
                    A.SetValue(str[i], 0, i);
                }
                return A;
            }
        }

Test Case : 

using (var file = new H5File("testwrite.h5"))
            {
                var ds = new H5Dataset("data", ILMath.rand(10,10));
                file.Add(ds);

                string teststr = "Test string";
                ILArray<char> charStr = ConvertStringToArray(mystr);
                ds.Attributes.Add(new H5Attribute("mystring",charStr));

                //Read back the dataset and its attributes
                var group = file.Find<H5Dataset>("data").First();
                ILArray<Char> storedData = group.Attributes["mystring"].Get<Char>();
            }
Neelima
  • 243
  • 1
  • 8
  • 'can not read' mean Exception for the VLEN String. The test.h5 in the link is create with an other program. I can it read without problems in matlab and hdfview. I search a library as replacement of hdf5dotnet, which is only a simple wrapper of the C Api and difficult to handle. – Pete Aug 21 '14 at 12:39
  • 1
    Datatypes of variable length are not supported in ILNumerics. We do not have urgent plans to do so. VL data come with [several drawbacks](http://www.hdfgroup.org/HDF5/doc/TechNotes/VLTypes.html) and most the time one can prevent from them. It would surely be good to be at least able to read them, though. – Haymo Kutschbach Aug 21 '14 at 14:30
  • Your hdf5 lib is awesome. For the most of my data I can use it. The vlen strings in attributes can be changed to fixed len strings. For vlen and compound dataset own extension methods should be possible. – Pete Aug 26 '14 at 05:53
  • The problem with the replacement is, that char[] is no string. In HDFView an so written string is only an byte / char array: "MyspecialType" >> 77,121,115,112,101,99,105,97,108,84,121,112,101. not very usefull. The same problem is in MATLAB. I have to build the string from byte Array. – Pete Aug 28 '14 at 08:25
  • HDF5 is a universal file format. And string is also a very common dataype. It should be easy read/writable by all other programs (Matlab, R, HDFView, Python ...). Not with the construct of byte array. – Pete Aug 28 '14 at 08:35