0

Am using Azure Synapse in combination with jupyter notebooks: enter image description here

Many of my jupyter notebooks import some custom python scripts like the util-import: enter image description here

However, there's no option hold *.py files in Azure Synapse. Always when i use the import functionalty, the *.py is transformed to a notebook (On laptop it was util.py, after Synapse import it's a notebook):

enter image description here

How can custom *.py files be used in Azure Synapse Notebooks, without transforming them from *.py to notebook?

R. Maier
  • 340
  • 2
  • 13

1 Answers1

1

There are several options for adding python to Synapse. You can manage them at the workspace, pool, or session level. The method I've had the most success with is loading from PyPi to a Spark Pool, but you can also upload Wheel or JAR files to your workspace and reference those in your notebooks. One caveat on using PyPi is that if the package has a dependency on C library packages, Synapse won't load it into your pool.

Joel Cochran
  • 7,139
  • 2
  • 30
  • 43
  • While I appreciate the response, I'd like to add a note of caution: Importing a local Python file stored alongside the notebooks for seamless code sharing isn't straightforward. While the methods you mentioned (workspace, pool, session, and PyPi) can be used, they often introduce significant overhead. This can be especially challenging when working with a utility file that's still under development and hasn't reached maturity or stability. – jeromerg Aug 29 '23 at 16:42