1

How is it possible to handle backslash as is with named_struct?

I am trying to achieve the return value of: {"test": "DE844\/374"}

a) with this: SELECT named_struct('test', "DE844\/374");

return value is: {"test": "DE844/374"}

b) with this: SELECT named_struct('test', "DE844\\/374");

return value is: {"test": "DE844\\/374"}

notNull
  • 30,258
  • 4
  • 35
  • 50
user14372101
  • 11
  • 2

2 Answers2

0

I think only way to get \ backslash character is to use decode() function by using unicode(\u005C) character sets.

Example:

from pyspark.sql.functions import *
df = spark.createDataFrame([('a',)],['a'])
df.withColumn("str",(expr('concat("DE844",decode("\\u005C", "utf-8"),"/374")'))).show(10,False)
#+---+----------+
#|a  |str       |
#+---+----------+
#|a  |DE844\/374|
#+---+----------+
notNull
  • 30,258
  • 4
  • 35
  • 50
0

You need to escape the \ twice:

spark.sql("select named_struct('test', 'DE844\\\\/374' ) as a ").show()

#+------------+
#|           a|
#+------------+
#|[DE844\/374]|
#+------------+
philantrovert
  • 9,904
  • 3
  • 37
  • 61