2

I'm trying to use custom transform using a column at Wrangler Plugin.

Is there any documentation where I can find the list of functions to apply the custom transform? Also for a specific case, I wanna replace the value of a column based on IF-ELSE condition (or multiple cases). Is there any way of it?

Kolban
  • 13,794
  • 3
  • 38
  • 60
Ashish Balhara
  • 63
  • 1
  • 5
  • For custom cases like this its better to use `Dynamic Spark` plugin that executes user provided spark code written in scala. It can be used to transfrom RDDs with full access to all Spark features. – Neelesh Oct 18 '19 at 06:18

2 Answers2

2

The custom transform supports JEXL, so you can find a list of functions to apply here: JEXL syntax. See the Conditional section of that page for information on how to do an if-else.

Ali Anwar
  • 431
  • 2
  • 8
  • I've tried it with JEXL syntax, initially multiline expression, but later I found that single line expressions are the seemingly only compatible one. – Ashish Balhara Apr 29 '19 at 06:29
1

You can use any of the Wrangler directives in a custom transform. The GitHub page for the plugin has a comprehensive list with examples. As Ali Anwar mentioned, directives support JEXL syntax. For an example have a look at set-column directive.

Tony Hajdari
  • 11
  • 1