My pipeline looks similar to the following:
parDo return list per processed line | beam.io.WriteToText
beam.io.WriteToText adds a new line after each list element. How can I remove this new line and have the values separated by comma so I will be able to build CSV file
Any help is very appreciated! Thanks, eilalan
To remove the newline char, you can use this:
beam.io.WriteToText(append_trailing_newlines=False)
But for adding commas between your values, there's no out-of-the-box feature on TextIO to convert to CSV. But, you can check this answer for a user defined PTransform that can be applied to your PCollection in order to convert dictionary data into csv data.
