Evening!
I'm with the following error in my code and can't understand exactly what should i do to solve it:
File "C:\Spark\python\pyspark\sql\dataframe.py", line 804, in count
return int(self._jdf.count())
File "C:\Users\ariel\AppData\Roaming\Python\Python39\site-packages\py4j\java_gateway.py", line 1321, in __call__
return_value = get_return_value(
File "C:\Spark\python\pyspark\sql\utils.py", line 190, in deco
return f(*a, **kw)
File "C:\Users\ariel\AppData\Roaming\Python\Python39\site-packages\py4j\protocol.py", line 326, in get_return_value
raise Py4JJavaError(
py4j.protocol.Py4JJavaError: <unprintable Py4JJavaError object>
During handling of the above exception, another exception occurred:
response = connection.send_command(command)
File "C:\Users\ariel\AppData\Roaming\Python\Python39\site-packages\py4j\clientserver.py", line 539, in send_command
raise Py4JNetworkError(
py4j.protocol.Py4JNetworkError: Error while sending or receiving
ConnectionRefusedError: [WinError 10061] Nenhuma conexão pôde ser feita porque a máquina de destino as recusou ativamente
My inputs are less then 10g, and even turning my default spark config to use 10g it is not working.
I already reinstalled spark and Py4J and is not working.
This error occurs when i was just trying to do a count on a Spark Dataframe after aggregation!I happens too when i was trying to convert the Spark dataframe to a Pandas Dataframe.
