Pipe delimiter in .bat file

Question

I am trying to export a table from BigQuery to Google CLoud Storage (GCS) using bq command in a bat file. The command has a parameter to export as a Pipe delimited file. The command works from the CLI as it is but when I run from a bat file using the call command it opens the command but does not execute it. Note: Same operation runs without the pipe delimiter parameter.

Any idea why?

call bq extract -F^^^| my-project:my-dataset.mytable gs://myfolder/myfile.csv

This is the only line I have in my bat file. when I run from command line it works:

C:\bq extract -F^^^| my-project:my-dataset.mytable gs://myfolder/myfile.csv

Output:

Waiting on bqjob_r01419067_00000169eed1d868_1 ... (0s) Current status: DONE

When I run the bat file, C:\test.bat

Output:

C:\bq extract -F^| my-project:my-dataset.mytable gs://myfolder/myfile.csv C:\

Why are you using CALL? Why are you triple escaping the pipe delimiter? With some programs I use it is as simple as just quoting the delimiter you want to use. `-F "|"`. Using CALL will double all carets in the command. — Squashman, Apr 05 '19 at 18:10
Hi @Squashman, I am using CALL because I have more commands to add and if I don't use it, processing stop after the first command. Quotes are not working — Nishanthi Chezhian, Apr 05 '19 at 18:56
Did you try `-F"^|"`? I found another [question on StackOverFlow](https://stackoverflow.com/a/49911964/1417694) that said that does work. — Squashman, Apr 05 '19 at 18:59
Yes it works from the command line but not from bat file.C:\Users\myfolder>CALL bq extract -F"^|" my-project:mydataset.mytable gs://myfolder/Fact.csv The filename, directory name, or volume label syntax is incorrect. — Nishanthi Chezhian, Apr 05 '19 at 19:08
I don't use BigQuery so I don't know what is inside the `bq.bat` file that would be treating the carets incorrectly. — Squashman, Apr 05 '19 at 19:19
See the [explanation](https://stackoverflow.com/a/29620495/1417694) from @jeb — Squashman, Apr 05 '19 at 19:28
Wohoo! this works! thank you so much @squashman and @jeb!! set "caret=^" CALL bq extract -F"%%CARET%%|" my-project:my-dataset.mytable gs://myfolder/myfile.csv — Nishanthi Chezhian, Apr 05 '19 at 19:41

score 4 · Accepted Answer · answered Apr 05 '19 at 19:31

4

IF Jeb's solution is correct you should be able to change your batch file code to this.

set "caret=^"
call bq extract -F"%%CARET%%|" my-project:my-dataset.mytable gs://myfolder/myfile.csv

answered Apr 05 '19 at 19:31

Squashman

13,649
5
27
36

Pipe delimiter in .bat file

1 Answers1