0

How to convert below rank() partition by oracle sql query to equivalent pyspark sql.

SELECT department_id,
       last_name,
       salary,
       commission_pct,
       RANK() OVER (PARTITION BY department_id
                    ORDER BY salary DESC, commission_pct) "Rank"
FROM   employees
WHERE  department_id = 80;
MT0
  • 143,790
  • 11
  • 59
  • 117
Madhup Srivastava
  • 446
  • 1
  • 6
  • 18
  • Have you tried running this query in spark-sql? Did you get an error? The wrong output? Please provide a [mcve] – pault Nov 01 '19 at 14:29
  • There are no errors, looking for a way to re-write using UDF's or dataframe functions. – Madhup Srivastava Nov 01 '19 at 14:49
  • 2
    Possible duplicate of [How to use window functions in PySpark?](https://stackoverflow.com/questions/31857863/how-to-use-window-functions-in-pyspark) – pault Nov 01 '19 at 14:52
  • 1
    I have founded this: https://stackoverflow.com/questions/41661068/group-by-rank-and-aggregate-spark-data-frame-using-pyspark Maybe it will help... Also this article: https://databricks.com/blog/2015/07/15/introducing-window-functions-in-spark-sql.html Cheers! – VBoka Nov 02 '19 at 02:08

0 Answers0