0

Below is the configuration which worked till December 1st

  1. composer-1.11.2-airflow-1.10.6
  2. Python – 3.6
  3. 'dbt==0.17.0',
  4. 'google-cloud-storage',
  5. 'google-cloud-secret-manager==1.0.0',
  6. 'protobuf==3.12.2'

With the above configuration we are observing below error. ERROR: Cannot install dbt and protobuf==3.12.2 because these package versions have conflicting dependencies.\n\nThe conflict is caused by:\n The user requested protobuf==3.12.2\n dbt-bigquery 0.17.0 depends on protobuf<3.12 and >=3.6.0\n\nTo fix this you could try to:\n1. loosen the range of package versions you've specified\n2. remove package versions to allow pip attempt to solve the dependency conflict\n\nERROR: ResolutionImpossible

Post receiving this error message we have tried many version changes to resolve the above issue. Below listed are few of the fixes applied and errors we have observed for those fixes.

  1. Fix applied: downgraded protobuf version from 3.12.2 to 3.11.3 as per the above error Error: Requested docutils<0.16,>=0.10 from https://files.pythonhosted.org/packages/d4/12/6c3fd74a590c7327c98cae008c11d536029fa9cd7924de477e8cb8804186/docutils-0.15.1-post1.tar.gz#sha256=f33ddb723332c6d6b6d99731ee1fc0c35eb4044a2df5cca1c64c8aa78eaf22cb (from botocore<1.15,>=1.5.0->dbt-redshift==0.17.0->dbt==0.17.0) has different version in metadata: '0.15.1'

  2. Fix applied: Specified docutils version to 0.15.2 as to fix the above error Error: SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Generating %s..." % output)?\n ----------------------------------------\nERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

  3. Fix applied: Tried upgrading python version from 3.6 to 3.8. Error: The path python3.8 (from --python=python3.8) does not exist.

  4. Fix applied: Removed package versioning for above libraries to allow pip attempt to solve the dependency conflict. Error: Command errored out with exit status 1:\n command: /tmp/venv__vuo821/bin/python3.6 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ghz6thkq/protobuf_a5303ec5c2df4986bb9b05948ae30052/setup.py'"'"'; file='"'"'/tmp/pip-install-ghz6thkq/protobuf_a5303ec5c2df4986bb9b05948ae30052/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-gi38587q\n cwd: /tmp/pip-install-ghz6thkq/protobuf_a5303ec5c2df4986bb9b05948ae30052/\n Complete output (6 lines):\n Traceback (most recent call last):\n File "", line 1, in \n File "/tmp/pip-install-ghz6thkq/protobuf_a5303ec5c2df4986bb9b05948ae30052/setup.py", line 52\n print "Generating %s..." % output\n ^\n SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Generating %s..." % output)?\n ----------------------------------------\nERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output

  5. Fix applied: Upgraded dbt to 0.17.2 Error: Received SIGTERM. Terminating subprocesses.

  6. Fix applied: Tried downgrading setuptools version to 49.6.0 Error: SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Generating %s..." % output)?\n ------------------------------- ---------\nERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

  7. Fix applied: upgraded Composer and Airflow versions to composer-1.13.2-airflow-1.10.9 Error: Got similar errors with above configuration combinations

user2640679
  • 54
  • 3
  • Hello, what is your question? Do you need the python requirements which would work? – fpajot Dec 09 '20 at 10:19
  • airflow 1.10.13 is yanked. Do not upgrade to it. https://pypi.org/project/apache-airflow/#history Airflow 1.10.14 should be released very soon. – Elad Kalif Dec 09 '20 at 12:09
  • Did you try what the error message suggested? "remove package versions to allow pip attempt to solve the dependency" – rmesteves Dec 10 '20 at 11:35

1 Answers1

0

Which pip version are you using? If you upgrade pip to a newer version, please note that the dependency resolution algo has fundamentally changed since pip@20.3: https://pip.pypa.io/en/stable/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020. Backtracking will happen to ensure there are no conflict dependencies: https://pip.pypa.io/en/stable/user_guide/#dependency-resolution-backtracking.

Stick to an older pip version (<20) may work in your case

Alan
  • 596
  • 5
  • 18