3

Does anyone know how to convert a string to date in Data Fusion so that it writes to the target as 'Date' instead of string?

We are using Data Fusion to consume a csv from GCS (Google Cloud Storage). Data Fusion detects all fields as string, we'd like change/convert some of those fields that are actually dates as a Date into BigQuery (target). Is this performed as part of the wrangling or is it one of the transform options in the pipeline? Thank you.

TechNewbie
  • 164
  • 2
  • 15

3 Answers3

2

You can use the Wrangler transform to do so. In the Wrangler UI, you can use Parse -> Simple Date in the column drop down menu to convert a String type column into Date.

Terence Yim
  • 134
  • 5
2

For this specific date format, the Wrangler Transform directive would be:

parse-as-simple-date date_field_dt yyyyMMdd
set-column date_field_dt date_field_dt.toLocalDate()

The second line is required if the destination is of type Date.

Skip empty values:

set-column date_field_dt empty(date_field_dt) ? date_field_dt : date_field_dt.toLocalDate()

References:

https://github.com/data-integrations/wrangler/blob/develop/wrangler-docs/directives/parse-as-simple-date.md

https://github.com/data-integrations/wrangler/blob/develop/wrangler-docs/directives/parse-as-date.md

https://cdap.atlassian.net/wiki/spaces/KB/pages/208076805/Perform+date+transformations+in+Wrangler

xgMz
  • 3,334
  • 2
  • 30
  • 23
0

Below is a picture where you can find the option you are looking for:

enter image description here

סטנלי גרונן
  • 2,917
  • 23
  • 46
  • 68
MarSch
  • 1
  • 2
  • 1
    Please put the gist of your advice in words - 2020 search engines aren't quite up to finding text in pixel rasters, and an external image hyperlink might go stale. – greybeard Sep 07 '20 at 16:34