Compressing files through vroom and piping to pixz

Question

I would like to write the mtcars data.frame to file using compression (xz in my case) using the vroom package and pixz but I can't get it to write a file. It complains that pixz does not exist even though it is installed locally.

According to the vroom vignette (https://vroom.r-lib.org/articles/vroom.html): To compress using a single thread:

vroom_write(mtcars, "mtcars.tsv.xz")

to compress using a parallel implementation and piping to pixz

vroom_write(mtcars, pipe("pixz > mtcars.tsv.xz"))


sh: pixz: command not found

Could be related to `$PATH` variable in your shell environment. — crestor, Aug 11 '21 at 12:20
Ah ok, I should say I am on macOS Big Sur, where which pixz produces: /opt/homebrew/bin/pixz — HCAI, Aug 11 '21 at 13:03
Then try `vroom_write(mtcars, pipe("opt/homebrew/bin/pixz > mtcars.tsv.xz"))`. — crestor, Aug 11 '21 at 13:19
`pipe("/opt/homebrew/bin/pixz > mtcars.tsv.xz"))` (forgot the inicial "/") — crestor, Aug 11 '21 at 13:55
perfect, thank you! So my understanding is that pixz is not my PATH. Is there a straightforward way of making it "visible"? — HCAI, Aug 12 '21 at 08:09
https://osxdaily.com/2014/08/14/add-new-path-to-path-command-line/ — crestor, Aug 12 '21 at 08:23

score 1 · Accepted Answer · answered Aug 12 '21 at 08:26

1

Check that the pixz command can be found.

You can express the full path ("/opt/homebrew/bin/pixz") or add to the default paths (variable $PATH). https://osxdaily.com/2014/08/14/add-new-path-to-path-command-line/

answered Aug 12 '21 at 08:26

crestor

1,388
8
21

Compressing files through vroom and piping to pixz

1 Answers1