1

I have a script which scans an email inbox for specific emails. That part's working well and I'm able to acquire the data I'm interested in. I'd now like to take that data and add it to a Django app which will be used to display the information.

I can run the script on a CRON job to periodically grab new information, but how do I then get that data into the Django app?

The Django server is running on a Linux box under Apache / FastCGI if that makes a difference.

[Edit] - in response to Srikar's question When you are saying " get that data into the Django app" what exactly do you mean?...

The Django app will be responsible for storing the data in a convenient form so that it can then be displayed via a series of views. So the app will include a model with suitable members to store the incoming data. I'm just unsure how you hook into Django to make new instances of those model objects and tell Django to store them.

jotik
  • 17,044
  • 13
  • 58
  • 123
Jon Cage
  • 36,366
  • 38
  • 137
  • 215

8 Answers8

3

I think Celery is what you are looking for.

keithxm23
  • 1,280
  • 1
  • 21
  • 41
  • How does that help me insert data into a DJango app? As far as I can see that would just replace the CRON job or am I missing something? – Jon Cage Jul 05 '12 at 17:56
  • Ah, okay makes more sense with some background: http://www.slideshare.net/idangazit/an-introduction-to-celery – Jon Cage Jul 05 '12 at 18:01
  • More docs here: http://celery.github.com/celery/django/first-steps-with-django.html – Jon Cage Jul 05 '12 at 18:05
  • Seems like this might be overkill when I only have a single server and one task that can be done sequentially? Could you elaborate on how I would hook my email-fetch into celery to post the information into DJango's backend? – Jon Cage Jul 06 '12 at 15:05
2

You can write custom admin command to load data according to your need and run that command through cron job. You can refer Writing custom commands

You can also try existing loaddata command, but it tries to load data from fixture added in your app directory.

Rohan
  • 52,392
  • 12
  • 90
  • 87
  • That looks promising. What command would you use in CRON to call the command? – Jon Cage Jul 05 '12 at 17:58
  • @JonCage You can refer http://stackoverflow.com/questions/1601153/django-custom-command-and-cron on how to run command from cron. – Rohan Jul 06 '12 at 04:04
1

I have done the same thing.

Firstly, my script was already parsing the emails and storing them in a db, so I set the db up in settings.py and used python manage.py inspectdb to create a model based on that db.

Then it's just a matter of building a view to display the information from your db.

If your script doesn't already use a db it would be simple to create a model with what information you want stored, and then force your script to write to the tables described by the model.

Charles Crofford
  • 33
  • 5
1

Forget about this being a Django app for a second. It is just a load of Python code.

What this means is, your Python script is absolutely free to import the database models you have in your Django app and use them as you would in a standard module in your project.

The only difference here, is that you may need to take care to import everything Django needs to work with those modules, whereas when a request enters through the normal web interface it would take care of that for you.

Just import Django and the required models.py/any other modules you need for it work from your app. It is your code, not a black box. You can import it from where ever the hell you want.

EDIT: The link from Rohan's answer to the Django docs for custom management commands is definitely the least painful way to do what I said above.

Tom Manterfield
  • 6,515
  • 6
  • 36
  • 52
0

When you are saying " get that data into the DJango app" what exactly do you mean?

I am guessing that you are using some sort of database (like mysql). Insert whatever data you have collected from your cronjob into the respective tables that your Django app is accessing. Also insert this cron data into the same tables that your users are accessing. So that way your changes are immediately reflected to the users using the app as they will be accessing the data from the same table.

Srikar Appalaraju
  • 71,928
  • 54
  • 216
  • 264
0

Best way?

Make a view on the django side to handle receiving the data, and have your script do a HTTP POST on a URL registered to that view.

You could also import the model and such from inside your script, but I don't think that's a very good idea.

cha0site
  • 10,517
  • 3
  • 33
  • 51
  • Seems inefficient to me to go through the webserver to do this rather than hook into the back end some how? – Jon Cage Jul 05 '12 at 17:42
0

Have your script send an HTTP Post request like so. This is the library Requests

>>> files = {'report.xls': open('report.xls', 'rb')}

>>> r = requests.post(url, files=files)
>>> r.text

then on the receiving side you can use web.py to process the info like this

x = web.input()

then do whatever you want with x

On the receiving side of the POST request import web and write a function that handles the post

for example 

    def POST(self):

        x = web.input()
SJP
  • 1,330
  • 12
  • 21
  • What are requests and web in your snippet above? Can you provide a fuller example? – Jon Cage Jul 05 '12 at 17:52
  • Requests docs can be found http://docs.python-requests.org/en/latest/user/quickstart/#more-complicated-post-requests and the web.py for handling the requests is found at http://webpy.org/ I just wrote a program doing basically the exact same thing you are trying to do, using these 2 libs – SJP Jul 05 '12 at 18:27
  • So web.py is a lightweight server that handles POST/GET requests? That's not what I want at all - I have DJango on the server to receive the data, I'm just not sure how I call the relevent bits of DJango to import the data... – Jon Cage Jul 05 '12 at 22:08
0

If you dont want to use HTTP to send messages back and forth you could just have the script write the email info to a .txt file and then have your django app open the file and read it.

EDIT:

You could set your CRON job to read the e-mails at say 8AM then write it to a text file info.txt. The in your code write something like 

import time
    if '9' == time.strftime("%H"):
        file = open(info.txt)
        info = file.read()

that will check the file at 9AM untill 10AM. if you want it to only check one time just add the minutes too the if statement as well.

SJP
  • 1,330
  • 12
  • 21
  • 1
    How would you trigger the DJango app to read the files? I want it to do the check periodically and fill in the information automatically. How would you trigger DJango to read those .txt files? – Jon Cage Jul 05 '12 at 22:09
  • 1
    That doesn't help - where do you call that from? A model? A view? – Jon Cage Jul 06 '12 at 15:03
  • Is your django app on the same machine as the script that is going through the emails? If so then put it on the back end of your django app where you need the information from the email. It would be a model. If you could include some more info on how your app works it might be easier to explain. The solution to your problem is really only a few lines of code. If you want to get real technical with celery thats an option but you really dont need to – SJP Jul 06 '12 at 16:36
  • That's correct; django app and email collector are on the same server. The email collector is just pure python so could be incorporated into the django app (which doesn't exist yet). The details are sketchy because all I have right now is the email script which currently sticks the information into an sql alchemy fronted DB. I want to stick the data into this new django app so that I can play with it and display the information in some more convenient formats. The bit I'm stuck on is how to trigger the script or how to incorporate it into the django app... – Jon Cage Jul 06 '12 at 17:11