1

I'm having trouble with using impyla library on windows

I installed impyla library

pip install impyla

Error occured when I tried to import impyla libary in python code

from impala.dbapi import connect  # error occured
from impala.util import as_pandas
conn = connect(host='10.xx.xx.xx', database='xx_xx', port=21050)`

Traceback (most recent call last): ...

File "D:/test/test.py", line 14, in from impala.dbapi import connect

File "C:\Anaconda3\lib\site-packages\impala\dbapi.py", line 28, in import impala.hiveserver2 as hs2

File "C:\Anaconda3\lib\site-packages\impala\hiveserver2.py", line 32, in from impala._thrift_api import (

File "C:\Anaconda3\lib\site-packages\impala_thrift_api.py", line 73, in include_dirs=[thrift_dir])

File "C:\Anaconda3\lib\site-packages\thriftpy\parser__init__.py", line 30, in load include_dir=include_dir)

File "C:\Anaconda3\lib\site-packages\thriftpy\parser\parser.py", line 496, in parse url_scheme))

thriftpy.parser.exc.ThriftParserError: ThriftPy does not support generating module with path in protocol 'c'

when I tried to print include_dir, which was

D:/test\thrift

I just cannot import libray at all

help me

Hyungsoon Park
  • 11
  • 1
  • 5

2 Answers2

8

I had the same problem with thriftpy, the problem on windows is an absolute path is like C:\foo\bar.thrift

But, the way the thrift library parses the file, it detects the C: as if it were a protocol like http: or https:

Its pretty easy to workaround you just have to strip the first two characters from the path with a slice like path[2:]

Just slice when you call thriftpy.load or in the library file

File "C:\Anaconda3\lib\site-packages\thriftpy\parser__init__.py", line 30

path = "C:\foo\bar.thrift"
thrift.load(path[2:], module_name, include_dirs=include_dirs,
                   include_dir=include_dir)

OR

You can go a bit deeper and make the same change that I already submitted as a patch on the github page... perhaps it will be incorporated in the next version of thrift.

File "C:\Anaconda3\lib\site-packages\thriftpy\parser\parser.py", line 488

-    if url_scheme == '':
+    if len(url_scheme) <= 1:

My justification of why this change is valid is in the pull request. If its incorporated then you wont have to worry about making the same change again when you update the library. If not then just strip the two characters again.

Update:

  • Thriftpy Version 1: the parser fix is now on line 547: elif len(url_scheme) <= 1:

  • Thriftpy Version 2: the fix has already been merged.

Shogan Aversa-Druesne
  • 309
  • 2
  • 3
  • Dear [Shogan Aversa-Druesne](http://stackoverflow.com/users/6787429/shogan-aversa-druesne), I have the same problem, and I did your changes but no one helped me (I got the same error). Do you have any idea? Thank you!! – ragesz Sep 05 '16 at 13:02
  • 1
    OK, it works, I had to restart the kernel to load the current (modified) module. Thx for the solution!!! – ragesz Sep 05 '16 at 13:26
  • @Shogan Aversa-Druesne really really thank you. problem solved :) – Hyungsoon Park Nov 01 '16 at 05:51
  • 1
    I used the second fix and just modified the `if url_scheme == ''` to become `if url_scheme == '' or url_scheme == 'c':`. Works great – Alter May 23 '17 at 18:39
  • Looking at this answer a few years later, I am facing the same issue and finding different lines in both `.../thrifty/parser/__init__.py` and `parser.py`. If someone knows a way around this, please post. – Diego-MX Jun 16 '20 at 16:24
  • Search for url_scheme in parser.py. You should find the spot. It is currently line 547 on the github master. – Shogan Aversa-Druesne Jul 26 '21 at 05:40
2

I was encountering the same error with impyla on an Anaconda Python 3.6 distribution on Windows. Instead of installing using pip, I was able to get it working using: 

conda install -c anaconda impyla

https://anaconda.org/anaconda/impyla

atm
  • 1,684
  • 1
  • 22
  • 24