0

I'm trying to use prettytable module to print out data from csv file. But it failed with the following exception

Could not determine delimiter error for valid csv file 

>>> import prettytable
>>> with file("/tmp/test.csv") as f:
...     prettytable.from_csv(f)
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "build/bdist.linux-x86_64/egg/prettytable.py", line 1337, in from_csv
  File "/usr/lib/python2.7/csv.py", line 188, in sniff
    raise Error, "Could not determine delimiter"
_csv.Error: Could not determine delimiter

The CSV file:

input_gps,1424185824460,1424185902788,1424185939525,1424186019313,1424186058952,1424186133797,1424186168766,1424186170214,1424186246354,1424186298434,1424186376789,1424186413625,1424186491453,1424186606143,1424186719394,1424186756366,1424186835829,1424186948532,1424187107293,1424187215557,1424187250693,1424187323097,1424187358989,1424187465475,1424187475824,1424187476738,1424187548602,1424187549228,1424187550690,1424187582866,1424187584248,1424187639923,1424187641623,1424187774567,1424187776418,1424187810376,1424187820238,1424187820998,1424187916896,1424187917472,1424187919241,1424188048340,dummy-0,dummy-1,Total
-73.958315%2C 40.815569,0.0(nan%),1.0(100%),1.0(100%),0.0(nan%),1.0(100%),1.0(100%),0.0(nan%),1.0(100%),1.0(100%),0.0(nan%),1.0(100%),1.0(100%),0.0(nan%),1.0(100%),1.0(100%),1.0(100%),1.0(100%),1.0(100%),0.0(0%),0.0(nan%),0.0(0%),0.0(nan%),0.0(0%),0.0(nan%),0.0(nan%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),13.0 (42%)
-76.932984%2C 38.992186,0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),1.0(100%),0.0(nan%),1.0(100%),0.0(nan%),1.0(100%),0.0(nan%),0.0(nan%),1.0(100%),1.0(100%),1.0(100%),1.0(100%),1.0(100%),1.0(100%),1.0(100%),1.0(100%),0.0(nan%),1.0(100%),1.0(100%),0.0(nan%),1.0(100%),1.0(100%),0.0(nan%),1.0(100%),1.0(100%),0.0(nan%),0.0(0%),17.0 (55%)
null_input-0,0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(nan%),0.0(0%),0.0(nan%),0.0(nan%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0  (0%)
null_input-1,0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(nan%),0.0(0%),0.0(nan%),0.0(nan%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),0.0(0%),0.0(0%),0.0(nan%),1.0(100%),1.0  (3%)
Total,0.0(0%),1.0(3%),1.0(3%),0.0(0%),1.0(3%),1.0(3%),0.0(0%),1.0(3%),1.0(3%),0.0(0%),1.0(3%),1.0(3%),0.0(0%),1.0(3%),1.0(3%),1.0(3%),1.0(3%),1.0(3%),1.0(3%),0.0(0%),1.0(3%),0.0(0%),1.0(3%),0.0(0%),0.0(0%),1.0(3%),1.0(3%),1.0(3%),1.0(3%),1.0(3%),1.0(3%),1.0(3%),1.0(3%),0.0(0%),1.0(3%),1.0(3%),0.0(0%),1.0(3%),1.0(3%),0.0(0%),1.0(3%),1.0(3%),0.0(0%),1.0(3%),31.0(100%)

If you anyone can inform me how to workaround the problem or other alternative alternatives, it will be very helpful.

Martin Tournoij
  • 26,737
  • 24
  • 105
  • 146
kjee
  • 359
  • 5
  • 19
  • What does the CSV file look like? There is no single CSV standard, and there are many incompatibilities between different CSV formats. Python tries to guess the correct one, but in this case it can't for some reason... – Martin Tournoij Mar 24 '16 at 14:06
  • @Carpetsmoker thanks a lot for your reply. From the link, you can download my testcase. It should re-produce the problem. And I know what delimiter would be in this is is is a comma(,). Is there anyway that I can define it? Instead CSV module try to guess? – kjee Mar 24 '16 at 14:21

2 Answers2

1

According to pypi, prettytable is only alpha level. I could not find where you could give it the configuration to pass to the csv module. So in that case, you probably should read the csv file by explicitely declaring the delimiter, and build the PrettyTable line by line

pt = None  # to avoid it vanished at end of block...
with open('/tmp/test.csv') as fd:
    rd = csv.reader(fd, delimiter = ',')
    pt = PrettyTable(next(rd))
    for row in rd:
        pt.add_row(row)
Serge Ballesta
  • 143,923
  • 11
  • 122
  • 252
0

Got the same, working on some messier csv's. Ended up implementing fallback method with manual search

from string import punctuation, whitespace
from collections import Counter


def get_delimiter(self, contents: str):
    # contents = f.read()
    try:
        sniffer = csv.Sniffer()
        dialect = sniffer.sniff(contents)
        return dialect.delimiter
    except Error:
        return fallback_delimiter_search(contents)

def fallback_delimiter_search(contents: str) -> str:
    # eliminate space in case of a lot of text
    content_chars = list(filter(lambda x: (x in punctuation or x in whitespace) and x!=' ', contents))
    counts = Counter(content_chars)
    tgt_delimiter = counts.most_common(1)[0][0]
    return tgt_delimiter
yomajo
  • 106
  • 6