6

I want to read in several files (index.html, style.css, main.js) to create a JSON payload for upload.

I know with nodejs, I can begin to create what I want like so:

var fs = require('fs');
fs.readFile('index.html', 'utf8', function (err, data) {
  if (err) throw err;
  out = JSON.stringify({"html": data});
  console.log(out);
});

Though how do I do that with jq?

hendry
  • 9,725
  • 18
  • 81
  • 139

3 Answers3

3

Use the raw input (-R) command line option to read the input in as a string. Then you can build your json result. You'll want to slurp it too (-s) for multiline text files.

$ jq -Rs '{ html: . }' index.html

However, this only works for text files. If you have binary files, you would have to encode them first. You could use base64 to do so.

$ base64 -w0 image.jpg | jq -R '{ jpg: . }'
Jeff Mercado
  • 129,526
  • 32
  • 251
  • 272
  • That doesn't look right. http://s.natalian.org/2015-10-27/1445930412_1912x1036.png Also interested in doing this for several files. – hendry Oct 27 '15 at 07:20
  • Well, that's a binary file, you would have to encode it first. – Jeff Mercado Oct 27 '15 at 07:26
  • Slurp is indeed needed, but how does it work with additional files? e.g. `jq -Rs '{ html: ., js .}' index.html main.js` does not work. – hendry Oct 27 '15 at 11:46
  • @peak: Ah, yes, for the multiline text files. Thanks. – Jeff Mercado Oct 27 '15 at 17:34
  • @hendry: jq, is not really suited to combining multiple files. It can to some degree for certain things, but doesn't really scale to doing much more than a handful. Do you have a more complete example of the types of files you want to combine and how you want them combined? What you're asking for might be possible without additional external tools but how to deal with each type will vary. – Jeff Mercado Oct 27 '15 at 17:39
3

This ought to work for you (requires jq 1.5):

jq --null-input --raw-input \
  'reduce inputs as $line ({}; .[input_filename] += [$line]) | map_values(join("\n"))' \
  index.html style.css main.js

Here's the filter on its own. It's pretty simple:

reduce inputs as $line ({}; .[input_filename] += [$line])
| map_values(join("\n"))

Example:

$ cat test1.txt
foo
bar
baz

$ cat test2.txt
qux
quux
quuux

$ jq --null-input --raw-input \
  'reduce inputs as $line ({}; .[input_filename] += [$line]) | map_values(join("\n"))' \
  test1.txt test2.txt
{
  "test1.txt": "foo\nbar\nbaz",
  "test2.txt": "qux\nquux\nquuux"
}

P.S. If you don't mind a trailing newline you could do this instead:

reduce inputs as $line ({}; .[input_filename] += "\($line)\n")
Jordan Running
  • 102,619
  • 17
  • 182
  • 182
2

One way to handle multiple text files is illustrated by the following:

(jq -Rs . a.txt ; jq -sR . b.txt) | jq -s
[
  "1\n2\n",
  "3\n4\n"
]

So in your case you would do something like this:

(jq -Rs '{ html: . }' index.html; \
 jq -Rs '{ javascript: . }' main.js; \
 jq -Rs '{ css: . }' style.css) |\
 jq -s add

That is, convert each text file to a JSON string separately, and then pipe these strings to jq. This has the advantage of not requiring jq 1.5, but if you do have jq 1.5, then a solution using the filter inputs might be preferable.

peak
  • 105,803
  • 17
  • 152
  • 177
  • Close! Could you change your answer to `(jq -Rs '{ html: . }' index.html; jq -Rs '{ javascript: . }' main.js; jq -Rs '{ css: . }' style.css) | jq -s add`? That's what I found worked for me. – hendry Oct 28 '15 at 03:18