0

If I refrain from importing any libraries above, and instead create a straightforward function to generate a wheel file, everything works perfectly:

def codeversion():
    print(f"Package Imported Successfully")

However, when I include certain libraries, generate the wheel file, and upload into the synapse spark pools, I encounter errors.

from typing import Optional
from datetime import datetime
from delta import *

Here is the error message

Warning: you have pip-installed dependencies in your environment file , but you do not list pip itself as one of your conda dependencies.
Conda may not use the correct pip to install your packages and they may end up in the wrong place. Please add an explicit pip dependency.

here is what my setup files look like:

import setuptools

setuptools.setup(
    name = 'depackage',
    version='0.0.1',
    description="",
    author='',
    packages=setuptools.find_packages(),
    license="MIT",
    classifiers=[
        "License :: OSI Approved :: MIT License",
        "Programming Language :: Python :: 3.10",
        "Operating System :: OS Independent",
    ],
    extras_require={
        "dev": ["typing", "datetime","delta","pip"],
    },
    python_requires = '>= 3.10.0'
)
Umer
  • 25
  • 5

1 Answers1

0

I tried below setup.py code and it worked.

setup.py code:-

from setuptools import setup, find_packages

setup(
    name='package',  # Replace 'your_package_name' with the actual name of your package
    version='0.1',  # Replace with the desired version number
    author='package name',  # Replace with your name
    description='Package is py',  # Replace with a brief description of your package
    packages=find_packages(),
    install_requires=[
        'typing',
        'datetime',
        'delta',
    ],
)

An alternative to install packages in synapse are given here

Visit your synapse workspace > Manage > Workspace packages, download the delta pypi JAR file from here and upload it as a package in the workspace like below:-

enter image description here

enter image description here

enter image description here

Refer this additional document too for wheel

SiddheshDesai
  • 3,668
  • 1
  • 2
  • 11