Convert a list returned by a map() function (from the Pool class) to a Dictionary in Python

Question

I have a function in Python that is called by a map() function iterating over all the elements of a list which are sent as parameters. This function returns a dictionary and the map function is assigned to a dictionary, but the Python interpreter converts it into a list.

hotels_with_low_wifi_quality = pool.map(get_accomodation_wifi_rating, hotels_url_list)

So in the sentence above, hotels_with_low_wifi_quality is a dictionary and get_accomodation_wifi_rating is a function that returns a dictionary.

But if I print hotels_with_low_wifi_quality it outputs the following:

[None, None, None, None, None, None, None, None, None, {u'\nPins Platja\n': u'4,3'}, None, None, None, None, None] None

This is clearly a list. How can I convert it into a dictionary where the None values are ignored and the only element is ["Pins Platja"] = 4.3

Below is the get_accomodation_wifi_rating

def get_accomodation_wifi_rating(hotel):
page = requests.get(hotel)
soup = BeautifulSoup(page.text, 'lxml')
hotel_name = soup.find(id = "hp_hotel_name").text
reviews = soup.find(class_="review_list_score_breakdown_right")
if reviews is not None:
    wifi_tag = reviews.find(attrs={"data-question" : "hotel_wifi"})
    if wifi_tag is not None:
        wifi_rating = wifi_tag.find(class_="review_score_value")
        wifi_score = wifi_rating.text
        wifi_score_num = wifi_score.replace(",", ".")
        if float(wifi_score_num) < 7:
            hotels_with_low_wifi_quality[hotel_name] = wifi_score
            return hotels_with_low_wifi_quality

`list.map` does not do what you seem to think it does. It returns a new list by applying your function `get_accomodation_wifi_rating` to every element of `hotels_url_list`. Could you show us `get_accomodation_wifi_rating`? It seems to be returning `None` most of the time. — Olivier Melançon, Jan 01 '18 at 01:22
If get_accomodation_wifi_rating used somethwere else? I would recommend you change its return type as it is a bit akward like this. You seem to be populating a global dict then returning it. — Olivier Melançon, Jan 01 '18 at 01:44

score 3 · Accepted Answer · answered Jan 01 '18 at 04:04

It looks as though you want to be able to get all the of the poor wifi scores in parallel. Everyone is correct that it's a really bad idea to use a global dictionary. There is a little less verbose way to accomplish what you want:

Return a (tuple) consisting of (hotel name, score) in your get_accomodation_wifi_rating function:

return (hotel_name, wifi_score)
Use python's built-in filter to filter out your NoneTypes:

hotels_with_low_wifi_quality = dict(filter(None, returned_list))

Filter above without a supplied function evaluates each item in the list based on python's evaluation of the truthiness of that item; just something to keep in mind.

Olivier Melançon · Answer 2 · 2018-01-01T02:02:29.873

Python has a better syntax, called list comprehension, for mapping and filtering. You could achieve what you want like so

# We first map to get ratings
ratings = [get_accomodation_wifi_rating(name) for name in pool]

# We then filter the None out
ratings = [x for x in ratings if x is not None]

# We now have a list of `dict` which we want to merge
hotels_with_low_wifi_quality = {
    name: rating
        for d in ratings
        for name, rating in d.items()
}

Although, I want to point out that your function get_accomodation_wifi_rating has a return type that is kind of akward for what you want to do.

First of all, it seems to be populating a global dict and return it, which is a really bad idea.

Second, for the use your are making of it, I would simply return a name, rating tuple to be able to do the following.

hotels_with_low_wifi_quality = dict(
    x
    for x in [get_accomodation_wifi_rating(name) for name in pool]
    if x is not None
)

Convert a list returned by a map() function (from the Pool class) to a Dictionary in Python

2 Answers2