Stack Overflow
Stack Overflow

Questions tagged [jsonlines]

JSON Lines is a format for storing structured data that may be processed one record at a time. It is a convenient format for storing structured data that may be processed one record at a time. It works well with Unix-style text processing tools and shell pipelines.

This text format is documented at http://jsonlines.org/.

156 questions
1
vote
4 answers

jq: insert new objects while reading inputs from json file and bash stdout

I want to insert new json objects in between json objects using bash generated uuid. input json file test.json {"name":"a","type":1} {"name":"b","type":2} {"name":"c","type":3} input bash command uuidgen -r target output json {"id":…
Gabe
  • 226
  • 3
  • 13
1
vote
2 answers

Read JSON file with multiple objects inside Python

I am trying to read a json file in Python and convert it into a dataframe. The problem is that my json file has several json objects inside. The structure of my json is like this: {"Temp":"2,3", "OutsideTemp" : "3,4",...} {"Temp":"3,2",…
heyou
  • 13
  • 4
1
vote
1 answer

Converting slist to csv

A shell script that I run in IPython returns the following object: results = ['{"url": "https://url.com", "date": "2020-10-02T21:25:20+00:00", "content": "mycontent\nmorecontent\nmorecontent", "renderedContent": "myrenderedcontent", "id": 123,…
Tea Tree
  • 882
  • 11
  • 26
1
vote
3 answers

Parsing a jsonl file into a useful structure

I am importing a jsonl file from my hard drive and trying to get it into a usable format. Here is how I'm importing the data. train_data=[] with open("Documents/data/train.jsonl",'r',encoding='utf-8') as j: for line in j: …
user2355903
  • 593
  • 2
  • 8
  • 29
1
vote
2 answers

How can I optimize this Powershell script, converting JSON to CSV?

I have a very large JSON Lines File with 4.000.000 Rows, and I need to convert several events from every row. The resulted CSV File contains 15.000.000 rows. How can I optimize this script? I'm using Powershell core 7 and it takes around 50 hours to…
Martijn Olthuis
  • 61
  • 8
1
vote
0 answers

json format each record in a single line

I have a json that looks like this: { "d": { "results": [ { "__metadata": { "id": "12", "uri": "xyz", "type": "Result" }, "YEAR": "2018", "MOW": "0102", "UNIT":…
incnnu
  • 173
  • 3
  • 14
1
vote
2 answers

R is very slow reading in .jsonl files

I need to read .jsonl files in to R, and it's going very slowly. For a file that's 67,000 lines, it took over 10 minutes to load. Here's my code: library(dplyr) library(tidyr) library(rjson) f<-data.frame(Reduce(rbind,…
A.Vail
  • 15
  • 2
1
vote
2 answers

Python: finding duplicates in large jsonl file

I'm trying to find all json objects in my jsonl file that contain the same identifier value. So if my data look like: { "data": { "value": 42, "url": "url.com", "details": { "timestamp": "07:32:29", …
Faenatek
  • 13
  • 4
1
vote
1 answer

If statement based on value existing in jsonlines file

I have code that pulls over 400 PDFs off a website via Beautiful Soup. PyPDF2 converts the PDFs to text, which is then saved as a jsonlines file called 'output.jsonl'. When I save new PDFs in future updates, I want PyPDF to only convert the new PDFs…
GMB
  • 13
  • 3
1
vote
1 answer

Google Apps Script - How to stream JSON data into BigQuery?

In this reference https://developers.google.com/apps-script/advanced/bigquery, In order to load CSV data into BigQuery, they use: var file = DriveApp.getFileById(csvFileId); var data = file.getBlob().setContentType('application/octet-stream'); …
0xh8h
  • 3,271
  • 4
  • 34
  • 55
1
vote
1 answer

How to extract elements from a JSONL file with changing elements?

I want to extract "text" from the tokens in a JSONL file. If a label is present then I want to extract that as well. If it is not present then I want to insert "O" as a value for the label {"text":"This is the first…
bjornvandijkman
  • 194
  • 12
1
vote
2 answers

Java - Read (JSON) data from S3 object line by line

I have an S3 object that contains JsonL lines. I want to read this object line by line in Java so that I can recursively keep parsing each line (i.e, each json) into a POJO and eventually convert the object contents into a list of POJO. As I…
iammrmehul
  • 730
  • 1
  • 14
  • 35
1
vote
1 answer

jq filter to transform jsonlines into a single json map

Input: {"aa":["a1","a2"]} {"bb":["a1","b2"]} Wanted output (no worries about potentially duplicate keys): { "aa":["a1","a2"], "bb":["b1","b2"] } Many thanks!
type
  • 1,137
  • 2
  • 9
  • 16
1
vote
1 answer

How to solve CORS error consuming an API using oboe in Angular2 app?

I have API that returns data like…
char m
  • 7,840
  • 14
  • 68
  • 117
1
vote
1 answer

golang yaml support for jsonlines

I've been trying to get the go yaml package to parse a file with jsonlines entries. Below is a simple example with three options of data to be parsed. Option one is a multi-doc yaml example. Both docs parse ok. Option two is a two jsonline example.…
grahamjamesaddis
  • 43
  • 6
1 2 3
…
10 11