List comprehension to dictionary with multiple commands (Python, curser.execute)?

Question

I work with Python 3.6. I have connected to Hive using Pyhive and my cursor is cur.

Objective: Build a databases - tables map in a dictionary: dbname1 : list of tables within this db.

I have built it using for loop but I would like to transfer it to a dictionary comprehension please tell me what is wrong with my dict comprehension:

lst = extract_dbs_tbls(cur, s = 'show databases')

map_tbl_db = {}

for i in lst:
    cur.execute("use %s" % i)
    map_tbl_db['i'] = extract_dbs_tbls(cur, 'show tables')

Note: lst is a list of databases names like ['default', 'dwh', ...]

Basically what I want is the following dict comprehension:

{i:j for i in lst cur.execute('use %i' % i) j = extract_dbs_tbls(cur, s = 'show tables')}

Where extract_dbs_tbls:

def extract_dbs_tbls(cur, s = 'show tables'):

    "Return list of dbs / tables. Note: use before 'use db_name'."

    cur.execute(s)
    lst = cur.fetchall()

    return [j[0] for j in lst];

I don't think this is a valid way to do dict compression, `{i:j for i in lst}` is valid Python, but `cur.execute('use %i' % i) j = extract_dbs_tbls(cur, s = 'show tables')` after `lst` will break your code. — Aristu, Mar 21 '18 at 11:32

ndpu · Accepted Answer · 2018-03-21T11:51:24.027

1

To call cur.execute as additional operation in list comprehension you can use some dummy var and list (cur.execute is placed in list [ ] to create iterable with one element just to call contained function) that will not be used in result dict. And your j var is not needed and replaced by in place calling extract_dbs_tbls:

{i: extract_dbs_tbls(cur, s = 'show tables')
 for i in lst
 for dummy in [cur.execute('use %i' % i)]}

above is equivalent to your loop:

map_tbl_db = {}
for i in lst:
    cur.execute("use %s" % i)
    map_tbl_db[i] = extract_dbs_tbls(cur, 'show tables')

edited Mar 21 '18 at 11:51

answered Mar 21 '18 at 11:34

ndpu

22,225
6
54
69

ndpu, why [cur.execute('use %i' % i)]? I mean what the second for actually does? – steves Mar 21 '18 at 11:41
1

@steves second `for` needed to call `cur.execute` – ndpu Mar 21 '18 at 11:45
Is there any way to combine the cur.execute to combine use and show tables? – steves Mar 21 '18 at 11:47
It is quite tricky. In that case the original seems to me more readable. I would use _ instead of dummy. – Arpad Horvath -- Слава Україні Mar 21 '18 at 11:56
1

@steves `cur.execute` is calling (1 time) to each iteration of `i` loop... `extract_dbs_tbls` get new `cur` on each iteration of `i`, just like in your function... – ndpu Mar 21 '18 at 11:57

List comprehension to dictionary with multiple commands (Python, curser.execute)?

1 Answers1