0

I am very new in the IDL scene and I am struggling for hours with a problem and I hope you can help me:

So right now I am trying to read data from a table ("file.txt"). I would like, that every column is saved in a variable (I thought about using STRARR)

I found this tutorial: http://www.idlcoyote.com/tips/ascii_column_data.html

This is very useful, when you want to read numbers for every column, which works fine. This is the table from the tutorial above:

Tutorial table:

Experiment 01-14-97-2b9c
No. of Data Rows: 5
Temperature         Pressure          Relative Humidity
  20.43             0.1654                 0.243
  16.48             0.2398                 0.254
  17.21             0.3985                 0.265
  18.40             0.1852                 0.236
  21.39             0.2998                 0.293

Code:

OPENR, lun, "tutorial.txt", /GET_LUN
header = STRARR(3)
READF, lun, header

data = FLTARR(3, 5)
READF, lun, data
temperature = data(0,*)

print, data
print, temperature

Output data:

  20.4300     0.165400     0.243000
  16.4800     0.239800     0.254000
  17.2100     0.398500     0.265000
  18.4000     0.185200     0.236000
  21.3900     0.299800     0.293000

Output temperature:

  20.4300
  16.4800
  17.2100
  18.4000
  21.3900

Looks quite good, for numbers. But what about when I have strings with dates, times, but also numbers in it, like this:

My table:

Experiment 01-14-97-2b9c
No. of Data Rows: 5
Date              Start time             End time             Value
12-Feb-2002       05:08:10               06:08:30             20
08-Mar-2002       07:35:38               09:25:59             100
20-Jun-2002       12:30:35               16:15:18             5536
25-Jul-2002       04:02:06               07:02:58             5822
02-Aug-2002       23:30:25               23:55:22             456

The code above won't work anymore. When I am using this my_var= data(0,*), the whole data will be saved in variable my_var, of course because the data are no more looking as columns, but in a whole row.

FLTARR is setting this data

12-Feb-2002       05:08:10               06:08:30             20

to this result (of course because of FLTarr)

12.0000
5.00000
6.00000
20.0000

And STRARR is saving the data good in my_var, but without separating the columns.

What I want:

I would like to have every column in one variable, so that I can handle these variable data later in another code.

dates = data(0,*)
starts = data(1,*)
ends = data(2,*)
values = data(3,*)

print, starts

Output: Start time

05:08:10
07:35:38
12:30:35
04:02:06
23:30:25

(and also the rest of my variables)

I hope you can help here. Maybe I misunderstood something, if so please let me know it.

For any other suggestion or solution I would be grateful.

Thanks in advance!

DavidH
  • 415
  • 4
  • 21
TheHeroOfTime
  • 751
  • 4
  • 9
  • 23

1 Answers1

1

My suggestion would be to use STRSPLIT in some manner, either each line as you read it or all at the end.

Here is an example of doing it all at the end. First, read the data into a data array (ignoring the header array):

IDL> openr, lun, 'file.txt', /get_lun
IDL> header = strarr(3)
IDL> readf, lun, header
IDL> data = strarr(5)
IDL> readf, lun, data
IDL> free_lun, lun

Then split on whitespace:

IDL> tokens = strsplit(data, /extract)

And, finally, extract elements by position:

IDL> dates = (tokens.map(lambda(x: x[0]))).toarray()
IDL> starts = (tokens.map(lambda(x: x[1]))).toarray()
IDL> ends = (tokens.map(lambda(x: x[2]))).toarray()
IDL> values = (tokens.map(lambda(x: long(x[3])))).toarray()

You've got your values now:

IDL> help, dates, starts, ends, values
DATES           STRING    = Array[5]
STARTS          STRING    = Array[5]
ENDS            STRING    = Array[5]
VALUES          LONG      = Array[5]
IDL> print, values
          20         100        5536        5822         456

UPDATE: Make sure you have done

IDL> compile_opt strictarr

in scope before these commands.

UPDATE: To sort these arrays by value:

IDL> ind = sort(values)
IDL> values = values[ind]
IDL> dates = dates[ind]
IDL> starts = starts[ind]
IDL> ends = ends[ind]
mgalloy
  • 2,356
  • 1
  • 12
  • 10
  • Your solution looks quite good, but for some reason this error appears, when the map function it's been called: **"Keyword MAP not allowed in call to: IDL_OBJECT::GETPROPERTY"** Do you know why this error message is appearing? I think the **strsplit** works, there are no white spaces. It looks like this: `12-Feb-200205:08:1006:08:3020` `08-Mar-200207:35:3809:25:59100` `20-Jun-200212:30:3516:15:185536` `25-Jul-200204:02:0607:02:585822` `02-Aug-200223:30:2523:55:22456` – TheHeroOfTime Dec 28 '16 at 09:23
  • It's probably an IDL version issue. I'm not sure when map was added, but fairly recently. What version are you using? – mgalloy Dec 28 '16 at 15:16
  • My IDL version is 8.5, Build id: 20150707 Maybe it is also an issue of my OS. I am using Windows 10 right now. – TheHeroOfTime Dec 28 '16 at 15:28
  • Wow, with the command "**compile_opt strictarr**" it really worked now. But why does it work after the compile_opt? Can you also put this **compile_opt strictarr** to your solution answer, so that other people can see it? Thank you very much for your help! – TheHeroOfTime Dec 29 '16 at 08:42
  • I also have one little question: With this solution, do you know how to **sort** the list by the value of the column "value"? The other corresponding values must also follow the sorted "value" column. Is this right now even possible? – TheHeroOfTime Dec 29 '16 at 09:57
  • 1
    "compile_opt stricter" tells IDL to use only square brackets for indexing, parentheses are for calling functions. While this doesn't seem like the case here, it is generally good practice and makes the parsing simpler for IDL. I would recommend putting it in a startup file *and* including it in *every* routine you write. – mgalloy Dec 29 '16 at 16:00