0

I am testing pyflink on

  • os: centos7
  • flink version: flink-1.14.3
  • virtualenv python version: Python 3.6.8

pip list:

  • apache-beam 2.27.0
  • apache-flink 1.14.3
  • apache-flink-libraries 1.14.3
  • avro-python3 1.9.2.1
  • certifi 2021.10.8
  • charset-normalizer 2.0.11
  • cloudpickle 1.2.2
  • crcmod 1.7
  • dill 0.3.1.1
  • docopt 0.6.2
  • fastavro 0.23.6
  • future 0.18.2
  • grpcio 1.43.0
  • hdfs 2.6.0
  • httplib2 0.17.4
  • idna 3.3
  • mock 2.0.0
  • numpy 1.19.5
  • oauth2client 4.1.3
  • pandas 1.1.5
  • pbr 5.8.1
  • pip 21.3.1
  • protobuf 3.17.3
  • py4j 0.10.8.1
  • pyarrow 2.0.0
  • pyasn1 0.4.8
  • pyasn1-modules 0.2.8
  • pydot 1.4.2
  • pyflink 1.0
  • pymongo 3.12.3
  • pyparsing 3.0.7
  • python-dateutil 2.8.0
  • pytz 2021.3
  • requests 2.27.1
  • rsa 4.8
  • setuptools 59.6.0
  • six 1.16.0
  • typing-extensions 3.7.4.3
  • urllib3 1.26.8
  • wheel 0.37.1

I tried to run this command :

  • (virtualenv) [myuser@myvm flink-1.14.3] ./bin/flink run -py examples/python/table/word_count.py And got the following error:

Caused by: java.io.IOException: Failed to execute the command: python -c import pyflink;import os;print(os.path.join(os.path.abspath(os.path.dirname(pyflink.file)), 'bin')) output: Traceback (most recent call last): File "", line 1, in ImportError: No module named pyflink

I am sure pyflink package is already installed. Does anyone know why?

David
  • 103
  • 1
  • 9

2 Answers2

1

To install PyFlink, you only need to execute:

python -m pip install apache-flink

and make sure you have a compatible Python version (>= 3.5).

The problem may be the Python Virtual Environment， refer to https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/python/faq/#preparing-python-virtual-environment

Also may be you can add option '-pyexec venv.zip/venv/bin/python3' and have a try

ChangLi
  • 772
  • 2
  • 8
0

You have to check if pyflink is well installed (in your venv)

also check if you are running Flink

if no, start it with :

start-cluster.sh

here is full documentation about PyFlink: https://nightlies.apache.org/flink/flink-docs-master/docs/dev/python/overview/

Zak_Stack
  • 103
  • 8