Using argparse to open a json file?

Question

I recently just started using argparse and I'm trying to use it to open a json file among other things like outputting into a csv file.

import argparse
import os
import json
import csv
import pandas as pd

parser = argparse.ArgumentParser(description='Json to Csv')
parser.add_argument('file', type=argparse.FileType('r'))
#parser.add_argument('file', type=str)
args = parser.parse_args()

with args.file as json_data:

print(json_data)
#argparse_dict = vars(args)
#argparse_dict.update(json_data)


baseline_dict = {}
try:
    baseline_dict = json.load(json_data)
except:
    print("JSON Baseline File {}: Unable to LOAD")

results_dict = baseline_dict["results"]

"""with open(args.file) as json_data:
baseline_dict = {}
try:
    baseline_dict = json.load(json_data)
except:
    print("JSON Baseline File {}: Unable to LOAD")"""

#Turns the new dictionary into a dataframe df = pd.DataFrame(results_dict)

When I try to open the json file using argparse in terminal I get this error. I know it's pulling the json file because I can get it to run before calling it as json_data.

with args.file as json_data:
JSON Baseline File {}: Unable to LOAD
Traceback (most recent call last):

Not part of your current problem, but your whole `for` loop makes no sense. If you want the value for a key named `"results"`, just do `results_dict = baseline_dict["results"]` (or `results_dict = baseline_dict.get("results", {})` if the `"results"` key might not exist and you want to silently use an empty `dict` in that case) and remove all the code from `results_dict = {}` to `del (baseline_dict)`, which has multiple errors in it. — ShadowRanger, Jun 24 '20 at 14:45
Drop the `FileType` use; let `argparse` give you a filename (string), which you can then `with open` and load. With the modern preference for `with open`, `FileType` is obsolete. — hpaulj, Jun 24 '20 at 15:57
@hpaulj: I disagree; having `argparse` catch the exceptions and convert to a useful error message for you is super-convenient. Sure, you need to add `with` yourself, but for scripts (as opposed to libraries), you have top-level control so the issues are fairly limited. — ShadowRanger, Jun 24 '20 at 16:57
Note that it's the object *returned* by `open`, not the call to `open` itself, that lets the `with` statement ensure it gets closed. — chepner, Jun 24 '20 at 17:40
Trying both ways with open and without filetype or with filetype and just with returns: JSON Baseline File {}: Unable to LOAD Traceback (most recent call last): See edits above. — YellowJacket, Jun 24 '20 at 18:04
@YellowJacket: You left out the actual exception. Remove the `try`/`except`; let the exception propagate and crash the program, print the actual error message and traceback. All you've shown is that an exception occurs, but you've hidden all the useful information from it. As is, it could be caused by anything; the file might be in an encoding different from the default locale encoding, the file might not contain valid JSON, etc. — ShadowRanger, Jun 24 '20 at 23:40

score 1 · Answer 1 · answered Jun 24 '20 at 14:39

1

Because you used argparse.FileType as the argument type, args.file is already an open file handle, exactly as if it were returned by open. If you want to make sure it's closed deterministically (and to give it an unqualified name), all you need is:

with args.file as json_data:

or just:

with args.file:

if aliasing to a local name isn't needed.

answered Jun 24 '20 at 14:39

ShadowRanger

143,180
12
188
271

So when I originally opened the file without argparse I had no problem using json.load I did your suggestion which got rid of the previous error and used: with args.file as json_data: but json.load(json_data) doesn't work – YellowJacket Jun 24 '20 at 14:57
1

@YellowJacket: Never use the phrase "doesn't work" here without explaining (with [MCVE]); I have literally no idea how to help you when I have no idea what's wrong beyond "it's wrong". – ShadowRanger Jun 24 '20 at 16:58

score 1 · Answer 2 · answered Jun 24 '20 at 20:09

A script that can load a file two ways:

import argparse, json

parser = argparse.ArgumentParser()
parser.add_argument('--file', type=argparse.FileType('r'))
parser.add_argument('--name')
args = parser.parse_args()
print(args)

if args.file is not None:
    print(json.load(args.file))
    args.file.close()
    
if args.name is not None:
    with open(args.name) as f:
         print(json.load(f))

sample runs:

1307:~/mypy$ python3 stack62557562.py 
Namespace(file=None, name=None)
1307:~/mypy$ python3 stack62557562.py --file foo.json
Namespace(file=<_io.TextIOWrapper name='foo.json' mode='r' encoding='UTF-8'>, name=None)
{'foo': 12, 'bar': 'twelve'}
1307:~/mypy$ python3 stack62557562.py --name foo.json
Namespace(file=None, name='foo.json')
{'foo': 12, 'bar': 'twelve'}
1307:~/mypy$ python3 stack62557562.py --name foo.json --file foo.json
Namespace(file=<_io.TextIOWrapper name='foo.json' mode='r' encoding='UTF-8'>, name='foo.json')
{'foo': 12, 'bar': 'twelve'}
{'foo': 12, 'bar': 'twelve'}

I tried your solution exactly, and get this for both solutions. raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Should I make a new question on stackoverflow for this? — YellowJacket, Jun 25 '20 at 15:23
I get that error if I try a second `json.load` on an open file. The first consumes the whole file, leaving it positioned at the end. — hpaulj, Jun 25 '20 at 15:31

Using argparse to open a json file?

2 Answers2