0

I have a Python3 application which inserts movie information into a mongodb back-end (MongoDB Compass), and a separate nodejs application which returns it. this was working fine, however i cleared the DB out and re-ran the application, now when I insert a document with insert_one the structure returned is always lost:

query = { "Name": name, "Year": year, "Format": definition, "Extension": extension, "Filepath": filepath, "MD5Hash": filehash, "Subtitles": subtitles, "Date": date, "FileSize": size } result = db.Movies_test.insert_one(query)



first run returned data:

_id: 5b1470da91cb5a2240c57c43 Extension: "mkv" Format: "1080p" Filepath: "E:\Movies\10 Cloverfield Lane [2016] 1080p.mkv" Year: "2016" FileSize: 2953397031 MD5Hash: "baf1b857c3f2f4fe6d08161e814a64c2" Date: 2018-06-03 23:51:06.503 Subtitles: false Name: "10 Cloverfield Lane" UpdateTime: 2018-06-04 00:42:51.141

second run returned data:

_id: 5b80322f91cb5a05a809a70f Year: "2016" FileSize: 2953397031 Format: "1080p" Filepath: "E:\Movies\10 Cloverfield Lane [2016] 1080p.mkv" Date: 2018-08-24 17:28:31.067 Name: "10 Cloverfield Lane" Extension: "mkv" MD5Hash: "" Subtitles: false

I noticed a similar question here: { $t: “”, $v: “”} structure in MongoDB collections using pymongo however there is no solution, the user just switched to using DocumentDB.

I read that insert has an extra option insert(query, {'ordered': True}), but this also does not keep the data structure:

_id: 5b80373691cb5a10c42c5906 Subtitles: false Format: "1080p" FileSize: 2953397031 Extension: "mkv" Date: 2018-08-24 17:49:58.160 Name: "10 Cloverfield Lane" MD5Hash: "" Filepath: "E:\Movies\10 Cloverfield Lane [2016] 1080p.mkv" Year: "2016"

Any help would be great, and ideally I would prefer to insert in the original query order, instead of processing the output when its comes out of nodejs.

daemon24
  • 1,388
  • 1
  • 11
  • 23
JONAS402
  • 591
  • 1
  • 9
  • 18
  • Why is the order for the document fields important here? – Oluwafemi Sule Aug 25 '18 at 14:13
  • @OluwafemiSule as stated, the returned document is appended to a table, ideally I'd like to keep the structure the same so I don't have to rebuild the tables each time (as I said, when I destroyed the db and rebuilt it, the structure changes). I managed to work around by rebuilding the query before appending to the table but I would prefer not to have to do this. – JONAS402 Aug 25 '18 at 15:14
  • AFAICT, mongo python driver predictably returns document fields ordered as they were written. I wasn't able to reproduce this behavior using pymongo. I think that this issue here may be with the node driver. – Oluwafemi Sule Aug 25 '18 at 16:10
  • Strange, my python insert_one query ran the same each time but produced different structures within mongo compass, so i don't think it's a nodeJs issue. Can you confirm that running an insert query, destroying that database and re running produces the same structure each time? As it doesn't for me! – JONAS402 Aug 25 '18 at 16:46
  • I tried it and made a [gist](https://gist.github.com/osule/6c09b8d0e21bd1889da06a9e5f4e38fe). Unfortunately, I didnt get the same results as you did. Maybe you want to look in how compass presents the results. – Oluwafemi Sule Aug 25 '18 at 17:44
  • @OluwafemiSule I tried your gist, works fine for me too... I just noticed the hash on the 1st set of returned data I posted. I remember now I have a second hashing function that hashes and adds a updateTime field. I'm wondering if this is actually modified the document structure as well. My bad! But to sum up, the documents keep the structure during insertion by default? – JONAS402 Aug 25 '18 at 18:42
  • Maybe you can share the hashing function in your question and we can figure out whether it causes this effect. To answer your question about ordering, there isn't a lot of tests to show it to be true everytime that field ordering is preserved. I'm guessing that since dictionaries keep the field ordering in python3, this should be the case. – Oluwafemi Sule Aug 27 '18 at 18:36
  • @OluwafemiSule so the relevant line would be `result = db.Movies.find_one_and_update({"_id": document['_id']}, {"$set": {"MD5Hash": md5, "UpdateTime": update_time}}, return_document=ReturnDocument.AFTER)`. I'm not seeing anything that would alter the structure. My fix was to edit the statement in node before sending it back to the table, this isn't so bad. But still, would be nice to know what the deal is! – JONAS402 Aug 27 '18 at 18:59

0 Answers0