1

I'm trying to load the following dataset:

Afghanistan,5,1,648,16,10,2,0,3,5,1,1,0,1,1,1,0,green,0,0,0,0,1,0,0,1,0,0,black,green

Albania,3,1,29,3,6,6,0,0,3,1,0,0,1,0,1,0,red,0,0,0,0,1,0,0,0,1,0,red,red

Algeria,4,1,2388,20,8,2,2,0,3,1,1,0,0,1,0,0,green,0,0,0,0,1,1,0,0,0,0,green,white

...

Problem is it contains both integers and strings. I found some information on how to get out the integers only. But haven't been able to see if there's any way to get all the data.

My question is that possible ??

If that is not possible, is there then any way to find the numbers on each line and throw everything else away without having to choose the columns?

I need specifically since it seems I cannot use str2num on a whole line at a time.

Community
  • 1
  • 1
Drakthal
  • 193
  • 3
  • 14
  • I'm not following, what exactly is it that you want to do? Do you want to parse the dataset into numbers and strings, or just filter out the numbers from each line? – Eitan T Jan 13 '13 at 11:18
  • At the moment i just want to get the dataset into a state where i can work with it tbh. Since i cannot just use load like ive done in previous projects. "Because there are strings in this dataset" I dont know how to proceed. – Drakthal Jan 13 '13 at 11:23

1 Answers1

6

Almost anything is possible, you just have to define your goal accurately.

Assuming that your database is stored as a text file, you can parse it line by line using textread, and then apply regexp to filter only the numerical fields (this does not require having prior knowledge about the columns):

C = textread('database.txt', '%s', 'delimiter', '\n');
C = cellfun(@(x)regexp(x, '\d+', 'match'), C, 'Uniform', false);

The result here is a cell array of cell array of strings, where each string corresponds to a numerical field in a specific line.

Since the numbers are still stored as strings, you'd probably need to convert them to actual numerical values. There's a multitude of ways to do that, but you can use str2num in a tricky way: it can convert delimited strings into an array of numbers. This means that if you concatenate all strings in a specific line back into one string, and put spaces in between, you can apply str2num on all of them at once, like so:

C = cellfun(@(x)str2num(sprintf('%s ', x{:})), C, 'Uniform', false);

The resulting C is a cell array of vectors, each vector containing the values of all numerical fields in the corresponding line. To access a specific vector, you can use curly braces ({}). For instance, to access the numbers of the second line, you would use C{2}.

All the non-numerical fields are discarded in the process of parsing, of course. If you want to keep them as well, you should use a different regular expression with regexp.

Good luck!

Eitan T
  • 32,660
  • 14
  • 72
  • 109