0

I get data in JSON format from one API. I need to extract specific fields and insert them into a MySQL table. Before inserting them I'm doing some changes, for example :

for record in api_data:
  col1 = record["field1"][0]
  col2 = int(record["field2"][:4])
  col3 = ';'.join(record["field3"])
  col4 = datetime.strptime(record["field4"][:10],'%Y-%m-%d').strftime('%d %b %Y')
  col5 = ' '.join(record["field5"])

Some of these fields can be empty and that's why a need a try block. Is there a more elegant way than using try except block for each field individually?

I checked this question : A DRY approach to Python try-except blocks? but it didn't help me.

coLby
  • 25
  • 6
  • It depends on what you want to do when the field is empty. Using the `get` method instead of catching an exception raised by `__getitem__` may be sufficient (or even preferable). – chepner Nov 23 '21 at 13:49
  • E.g., `col1 = record.get("field1", [some_default_for_col1])[0]` – chepner Nov 23 '21 at 13:50
  • And also, what's the problem with the mentioned answer? Extracting a function seems OK in this case. If the issue is that not every field has an equal treatment, you could also pass a function as a parameter, defining what to do to the field. Such as `prepare(record1, lambda x: x.get("field1")[0])` – MatBBastos Nov 23 '21 at 14:07
  • @chepner that didn't work how i expected when the field value is empty. If the field is empty for col1 then the values should be '' and the script should advance to col2. – coLby Nov 23 '21 at 14:20
  • That's why I said "it depends". You haven't said anything about what the code should actually do. – chepner Nov 23 '21 at 14:21
  • If the field is empty then the column should also be empty and the script should go to the next column. – coLby Nov 23 '21 at 14:25
  • Then simply `col1 = record.get("field1", "")` could do the trick? That way if it doesn't find `field1` in `record`, it will return a default value (empty string). Being a list with a value, you could use `record.get("field1", [""])[0]` for the default to still be empty – MatBBastos Nov 23 '21 at 14:46
  • That will not work : it will give a list index out of range exception. – coLby Nov 23 '21 at 14:58
  • @coLby, have you tried it? It definitely won't give such an exception, unless the field is __not__ empty, but contains an empty list, which is not what you described (or what I understood, let's say) as "when the field value is empty" – MatBBastos Nov 23 '21 at 16:43
  • I assumed that it was self-evident that some fields would be empty lists. – coLby Nov 23 '21 at 19:02
  • Usually, when talking technical matters and asking for aid in something specific issues of which people have no prior knowledge, it's best to leave no room for assumption and give detail, describe more, thoroughly the environment. I quote what you said: "Some of these fields can be empty". This does __not__ imply empty lists as values. I, for example, assumed "empty" to mean no matching key in the dictionary – MatBBastos Nov 23 '21 at 19:36
  • That being said, if you might have empty lists in `field1`, for example, you can still use a similar approach, with `col1 = record.get("field1", [""])[:1]`. The slicing operator will not raise error if the list is empty, it will return all the elements from the zero-th to the first index (aka, the first element) – MatBBastos Nov 23 '21 at 20:06
  • So, all in all, I still don't see why the answer you mentioned is not adequate. It surely solves the question with smart use of lambda or predefined functions. If you don't agree, please add more detail to the question so as to pinpoint the issues and what you expect of a solution – MatBBastos Nov 23 '21 at 20:16
  • For field1 = ["value"], If you use col1 = record.get("field1", [""])[0] and the value is empty that will throw a list out of index exception , if you use col1 = record.get("field1", [""])[:1] then it will either return an empty list or the value in a list : ["value"] which is not what i want. – coLby Nov 24 '21 at 07:50
  • In that specific case, then, I believe you need to either check before getting the item, or catching the exception and then acting on it (which is preferable). There are 'one-line' alternatives, but I believe they defeat readability and really are not the best approach. In any case, here's 'one-line' `if` for that: `col1 = record.get("field1")[0] if record.get("field1", False) else ""` – MatBBastos Nov 24 '21 at 11:52
  • In the end i ended up with this solution : col1 = record["field1"][0] if record["field1"] else ''. Thanks for your help. – coLby Nov 24 '21 at 11:58
  • I'd still vouch for extracting a function and 'preparing' each field using it, then catching any exceptions inside the function (and logging it if relevant). It will be more readable and, therefore, easier to maintain. It will also take care of all the current fields (and future ones, if that happens) with a single move. If you want a code snippet of it, I can post an answer with that option. Otherwise, I suggest you [answer your own question](https://stackoverflow.com/help/self-answer) – MatBBastos Nov 24 '21 at 12:14

0 Answers0