0

I used the below to create a tar.gz file, and .gz file was created but tar file was not available. How to achieve the result?

PCollection<String> lines = pipeline.apply("To read from file",
    TextIO.read().from(<file path>));

lines.apply(TextIO.write()
    .to("C:\\Ddrive\\saveAllRequest1.txt")
    .withCompression(Compression.GZIP);

and how to tar multiple files.

Thanks in advance

Nowhere Man
  • 19,170
  • 9
  • 17
  • 42
sathiya raj
  • 35
  • 1
  • 5

1 Answers1

1

Creating a tar file is an inherently non-parallel operation. One option here is to use the Wait transform after your write with a DoFn that manually creates the tarball.

robertwb
  • 4,891
  • 18
  • 21