0

I am trying to understand what exactly Snowpark pushdown feature do? from the documentation it already looks like the code is getting executed directly on Snowflake rather than the external clusters. Also, is it possible for me to run the code on my cluster instead of Snowflake using the Snowpark?

Raphael Roth
  • 26,751
  • 15
  • 88
  • 145
Rahul
  • 47
  • 5

1 Answers1

1
  • Snowpark supports pushdown for all operations, including Snowflake UDFs.

That means the data operations (applying filters, transformations etc) are pushed to the Snowflake engine, the database handles this workload and Snowpark just deals with the rest.

  • Snowpark does not require a separate cluster outside of Snowflake for computations. All of the computations are done within Snowflake.

Snowpark runs on the Snowflake warehouses. You don't need to run your code on your own external cluster such as EMR etc. You can create a development environment but Snowpark is designed to run on Snowflake:

https://docs.snowflake.com/en/developer-guide/snowpark/python/setup.html

Gokhan Atil
  • 9,278
  • 2
  • 11
  • 24