0

I'm newbie in python + scrapy + alembic. I finding a way to insert some init data in alembic using bulk_insert function.

my code: (direct insert)

op.bulk_insert('table',
[
  { 
    'id' : 1,
    'website': 'http://example',
    'xpath_pagination': '',
    'xpath_product': '',
  },
  { 
    'id' : 1,
    'website': 'http://example',
    'xpath_pagination': '',
    'xpath_product': '',
  },
  { 
    'id' : 1,
    'website': 'http://example',
    'xpath_pagination': '',
    'xpath_product': '',
  },
])

And now, I want to change way insert from out site

hope code:

import initdata
...
op.bulk_insert('table', initdata.data)

My quesion: how to do this right way? (type file using in this case). How to insert module initdata.

Thanks

Tue Vo
  • 323
  • 2
  • 3
  • 14

1 Answers1

0

If initdata is a module of the form

data = [
   { 
       'id' : 1,
       'website': 'http://example',
       'xpath_pagination': '',
       'xpath_product': '',
   }
]

then you can call bulk insert as you show, with

import initdata
op.bulk_insert('table', initdata.data)

If, however, initdata is a text file, then you will need to put into an array of dictionaries of appropriate structure. In this case, it is naturally suited to being JSON.

enalicho
  • 570
  • 3
  • 6
  • I created a init module in root project. /init/initdata.py (\_\_init\_\_.py exits"). And try import at alembic (import init or import init.initdata). But I failed, cli display errors: "No module named init" – Tue Vo Apr 15 '15 at 17:16