OrderedDict of OrderedDict and storing data in YAML Issues

Question

So basically I have an app I'm making that has user data which I want to backup and load in the database. I'm storing the data in yml files. Now, a user has posts. Each post has a timestamp, text and tags. I want to use an ordereddictionary in order to retain order when I write the data in the YAML files. Currently, I'm doing something like this:

def get_posts(user):
    posts_arr = []

    for post in user.posts.all():
        temparr = OrderedDict()

        temparr['timestamp'] = post.created_at.strftime("%Y-%m-%d %H:%M %p")
        temparr['text'] = post.text
        temparr['tags'] = (',').join(list(post.tags.all().values_list('field',flat=True)))

        posts_arr.append(temparr)

    return posts_arr

As you can see, I'm using an array of orderectionaries and that I think is the reason my posts for each user are not ordered. How can I resolve this issue.

I am returning this posts_arr object to be stored within another ordereddictionary.

Also, I since the posts text is kind of nested and is a large block of text, I want to make sure that text is also stored in string literal block styles.

You don't need an `OrderedDict` for either of these. `temparr` can be an ordinary `dict`, because you don't (or shouldn't) care about the relative order of the `timestamp`, `text`, and `tags` fields, and `posts_arr` should just be a *list*. — chepner, Mar 12 '19 at 16:34
Unfortunately I do need to maintain the order as well in order to ensure readability for ease. If I don't do that and then all my posts will have irregular order of timestamp, text and tags — Hamza, Mar 12 '19 at 16:37
If you are using Python 3.7, the insertion order will be preserved in a regular `dict` as well. — chepner, Mar 12 '19 at 16:41
What exactly is not ordered? Can you provide a piece of your YAML file as it comes out, as well as it should come out. Does `user.posts.all()` iterate in the order you want? — Anthon, Mar 12 '19 at 16:54
Somewhat unrelated, what types are you dealing with that have `all` method and `values_list` methods? — chepner, Mar 12 '19 at 16:54
And what does the original YAML look like? Is `user.posts.all()` guaranteed to provide objects in the order they appear in the YAML serialization? — chepner, Mar 12 '19 at 16:55
Your last sentence points to the problem of using YAML as a *storage* format rather than a *interchange* format. The type of string literal just isn't important to the consumer of a YAML file, and isn't preserved. Normal YAML processors simply aren't concerned with preserving or generating a particular serialization format, as long as the data itself is preserved. — chepner, Mar 12 '19 at 16:59

score 0 · Answer 1 · answered Mar 12 '19 at 16:39

Use an ordinary dict (or OrderedDict if you really need to) for each post, and use a list for the collection of all posts. Once you do that, it's a short jump to using a list comprehension to define the return value directly.

def get_posts(user):
    return [{
                'timestamp': post.created_at.strftime("%Y-%m-%d %H:%M %p"),
                'text': post.text,
                'tags': ','.join(list(post.tags.all().values_list('field', flat=True)))
            } for post in user.posts.all()]

score 0 · Accepted Answer · answered Mar 12 '19 at 16:48

Basically, your issue is a misunderstanding on how ordered dictionaries work in python. The python documentation states that an OrderedDict is a:

dict subclass that remembers the order entries were added

https://docs.python.org/3/library/collections.html#module-collections

Personally, I'd recommend a list of dictionaries created from a pre-sorted list of posts. In this case, it would look something like this if we were to keep the majority of your code as-is:

def get_posts(user):
    posts_arr = []
    sorted_posts = sorted(user.posts.all(), key=(lambda post: post.created_at))  # Sorts the posts based on their created_at date
    for post in sorted_posts:
        temparr = dict()

        temparr['timestamp'] = post.created_at.strftime("%Y-%m-%d %H:%M %p")
        temparr['text'] = post.text
        temparr['tags'] = (',').join(list(post.tags.all().values_list('field',flat=True)))

        posts_arr.append(temparr)

    return posts_arr

You could use list comprehensions to build this list from the sorted one like chepner suggested, but I don't want to change too much.

OrderedDict of OrderedDict and storing data in YAML Issues

2 Answers2