0

scripts\loadDateData.py

import csv
import urllib.request
from database.models import locationData, dateData

def run():
    target = locationData.objects.get (pk = 1)
    url = target.resourceURL
    headers = {'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0'} #from blog.csdn.net
    req = urllib.request.Request(url = url, headers = headers) #from blog.csdn.net
    response = urllib.request.urlopen(req) #from stackoverflow
    lines = [l.decode('utf-8') for l in response.readlines()] #from stackoverflow
    reader = csv.DictReader(response) #from stackoverflow
    
    dateData.objects.all().delete()
    
    for row in reader:
        print (row)
        temp = dateData()
        temp.date = row ['As of date']
        temp.confirmedCase = int (row ['Number of confirmed cases'])
        temp.deathCase = int (row ['Number of death cases'])
        temp.name = target.name
        temp.save()

models.py

from django.db import models
from django.core.validators import MaxValueValidator, MinValueValidator
#from django.contrib.postgres.fields import ArrayField

# Create your models here.
class locationData (models.Model):
    locationID = models.AutoField (primary_key = True)
    name = models.CharField (max_length = 64)
    population = models.IntegerField (default = 0, validators = [MinValueValidator(0)])
    apiEndpoint = models.URLField (max_length = 256)
    resourceURL = models.URLField (max_length = 256) 
    
    
class dateData (models.Model):
    entryID = models.AutoField (primary_key = True)
    name = models.CharField (max_length = 64)
    date = models.DateField (auto_now = False, auto_now_add = False)
    confirmedCase = models.IntegerField (default = 0, validators = [MinValueValidator(0)])
    deathCase = models.IntegerField (default = 0, validators = [MinValueValidator(0)])

I am a beginner and I am working on a project about a web-based application to show updated covid case data of a specific location. And I need to get data from the public released CSV files to do so. So I have followed an online tutorial and there is what I have done so far.

  • created a scripts directory in the project directory
  • created an init file, which is empty in that directory
  • added django_extensions into registered application in setting.py
  • and created that loadDateData.py shown above

Now, the error previously mentioned is handled. And there's a new problem that the database still empty after I run the server. As there is no error message popped out and I am not so familiar with Django. I am not sure which part goes wrong. Is there anything that I am still missing?

  • Here's the link of the csv I am working on, http://www.chp.gov.hk/files/misc/latest_situation_of_reported_cases_covid_19_eng.csv –  Mar 20 '21 at 05:47

1 Answers1

0

Bro simply use pandas .

import pandas as pd
data=pd.read_csv("yourFilePath.csv")
#or
data=pd.read_excel("yourFilePath.excel")
and get your dataset from https://archive.ics.uci.edu/ml/index.php or from kaggle