Python convert dictionary to CSV

Question

I am trying to convert dictionary to CSV so that it is readable (in their respective key).

import csv
import json
from urllib.request import urlopen
x =0
id_num = [848649491, 883560475, 431495539, 883481767, 851341658, 42842466, 173114302, 900616370, 1042383097, 859872672]
for bilangan in id_num:

with urlopen("https://shopee.com.my/api/v2/item/get?itemid="+str(bilangan)+"&shopid=1883827")as response:
    source = response.read()

data = json.loads(source)
#print(json.dumps(data, indent=2))


data_list ={ x:{'title':productName(),'price':price(),'description':description(),'preorder':checkPreorder(),
             'estimate delivery':estimateDelivery(),'variation': variation(), 'category':categories(),
             'brand':brand(),'image':image_link()}}

#print(data_list[x])
x =+ 1

i store the data in x, so it will be looping from 0 to 1, 2 and etc. i have tried many things but still cannot find a way to make it look like this or close to this:

https://i.stack.imgur.com/WoOpe.jpg

try not to link to external sites because over time these links could break and if you want to post data post it directly not a picture. also it would be much better if you could just post the final version of your dictionary since your example cant be run without your full code. — AntiMatterDynamite, Jul 01 '18 at 06:21
the link actually show the image on what i'm trying to get. I cannot post image yet. I'm new to coding — Muhammad Shahmi, Jul 01 '18 at 06:58
I would probably do your task differently. It appears that you are not using the power of dictionary to gather your data, as I see functions everywhere. This site returns a beautiful JSON response that is easy to work with. Another point is, I would use pandas to have full control over my data. With it you can store data to different formats, e.g. flat files as CSV, noSQL(e.g. MongoDB) ,compressed pickles, HD5 etc. Posting example coding as another possible answer below. — Prayson W. Daniel, Jul 01 '18 at 11:16

score 3 · Accepted Answer · answered Jul 01 '18 at 06:22

3

Using DictWriter from the csv module

Demo:

import csv

data_list ={'x':{'title':'productName()','price':'price()','description':'description()','preorder':'checkPreorder()',
             'estimate delivery':'estimateDelivery()','variation': 'variation()', 'category':'categories()',
             'brand':'brand()','image':'image_link()'}}

with open(filename, "w") as infile:
    writer = csv.DictWriter(infile, fieldnames=data_list["x"].keys())
    writer.writeheader()
    writer.writerow(data_list["x"])

answered Jul 01 '18 at 06:22

Rakesh

81,458
17
76
113

i've tried it but only one data will show up if I change from x to 'x'. So, i have editted the code so that, you have a bettervview on what im trying to do. – Muhammad Shahmi Jul 01 '18 at 06:51
Can you post a sample of `data_list ` so that i can modify this code? – Rakesh Jul 01 '18 at 13:08

score 0 · Answer 2 · answered Jul 01 '18 at 06:44

I think, maybe you just want to merge some cells like excel do? If yes, I think this is not possible in csv, because csv format does not contain cell style information like excel. Some possible solutions:

use openpyxl to generate a excel file instead of csv, then you can merge cells with "worksheet.merge_cells()" function.
do not try to merge cells, just keep title, price and other fields for each line, the data format should be like:

first line: {'title':'test_title', 'price': 22, 'image': 'image_link_1'}

second line: {'title':'test_title', 'price': 22, 'image': 'image_link_2'}
do not try to merge cells, but set the title, price and other fields to a blank string, so it will not show in your csv file.
use line break to control the format, that will merge multi lines with same title into single line.

hope that helps.

Prayson W. Daniel · Answer 3 · 2018-07-01T11:32:02.137

If I were you, I would have done this a bit differently. I do not like that you are calling so many functions while this website offers a beautiful JSON response back :) More over, I will use pandas library so that I have total control over my data. I am not a CSV lover. This is a silly prototype:

import requests
import pandas as pd

# Create our dictionary with our items lists

data_list = {'title':[],'price':[],'description':[],'preorder':[],
             'estimate delivery':[],'variation': [], 'categories':[],
             'brand':[],'image':[]}

# API url
url ='https://shopee.com.my/api/v2/item/get' 

id_nums = [848649491, 883560475, 431495539, 883481767, 851341658,
          42842466, 173114302, 900616370, 1042383097, 859872672]
shop_id = 1883827

# Loop throw id_nums and return the goodies
for id_num in id_nums:
    params = {
         'itemid': id_num, # take values from id_nums 
        'shopid':shop_id}
    r = requests.get(url, params=params)

    # Check if we got something :)
    if r.ok:
        data_json = r.json()

        # This web site returns a beautiful JSON we can slice :)

        product = data_json['item']

        # Lets populate our data_list with the items we got. We could simply
        # creating one function to do this, but for now this will do
        data_list['title'].append(product['name'])
        data_list['price'].append(product['price'])
        data_list['description'].append(product['description'])
        data_list['preorder'].append(product['is_pre_order'])
        data_list['estimate delivery'].append(product['estimated_days'])
        data_list['variation'].append(product['tier_variations'])
        data_list['categories'].append([product['categories'][i]['display_name'] for i, _ in enumerate(product['categories'])])
        data_list['brand'].append(product['brand'])
        data_list['image'].append(product['image'])

    else:
            # Do something if we hit connection error or something.
            # may be retry or ignore
            pass



# Putting dictionary to a list and ordering :)
df = pd.DataFrame(data_list)
df = df[['title','price','description','preorder','estimate delivery',
         'variation', 'categories','brand','image']]

# df.to ...? There are dozen of different ways to store your data 
# that are far better than CSV, e.g. MongoDB, HD5 or compressed pickle

df.to_csv('my_data.csv', sep = ';', encoding='utf-8', index=False)

Python convert dictionary to CSV

3 Answers3

Linked