0

I have a motion capture file that has some information which looks like: enter image description here

I need the file to look like this: enter image description here

The difference is that i dont need any of the top parts, just starting at row 31 column b. The on line 31, there is the keywords like LFHD and under it is xyz positions. My code works with the modified version by reading the csv and getting a dictionary for each line:

filename = args
jointCSVs=[]
if filename != None:
    print "Trying to get file"
    joint_file = open(str(filename),"rb")
    reader = csv.DictReader(joint_file)
    print "Got File"
    for data in reader:
        jointCSVs.append(data)
    return jointCSVs

I cant find any good information on how to do more complex things with python and csv's, especially since with my code, only the first block is read in and it ends when there is the whitespace. I even tried this code that i found but it wouldnt open the first csv file, it just crashes: pyqt - populating QTableWidget with csv data

It takes too long to modify by hand, and i have lots of these files. Later I plan on making some more gui stuff that can edit the files and modify them or select parts which would be useful for many people dealing with mocap data. I couldnt find any libraries for complex data manipulation or easy methods for specifying rows and columns to remove from a file. Does this exist or can someone explain how to do this?

Thanks!

Community
  • 1
  • 1
user-2147482637
  • 2,115
  • 8
  • 35
  • 56

1 Answers1

1

I often use Numpy when working with data sources like you have. There are some powerful functions for reading file data into a Numpy array. Once in a numpy array the rows and columns are easy to manipulate. You could slice off the top 30 rows for example, manipulate the headers and write back out to a file again with a python script. As a quick example:

import numpy as np

data = np.loadtxt("input.csv", skiprows=32)

# play with data, remove 1st column and write back to disk...
data = data[:, 1:]

np.savetxt("output.csv", data, header='LFHDX,LFHDT')

Work out your new header and pass it into savetxt.

dwxw
  • 1,089
  • 1
  • 10
  • 17
  • that seems good but i get a few errors when trying to load the file with numpy. One is an error converting string to float when loading the mocap data. – user-2147482637 Jun 13 '13 at 00:09
  • What's the specific error? You may have to play with the dtype, or define converters to handle edge cases. Did you find [genfromtxt](http://docs.scipy.org/doc/numpy/reference/generated/numpy.genfromtxt.html#numpy.genfromtxt), that is more powerful for when reading complex data files? – dwxw Jun 13 '13 at 09:03