0

So, I think i'm going insane. I'm using Delorean to convert a string in a CSV field row[15] into a date, which I then pass into parse() and then .epoch. Here are my imports:

import sys
import os.path
import requests
import json
import csv
import glob2
import shutil
from time import sleep
from time import gmtime, strftime
from delorean import Delorean
from delorean import parse
from delorean import epoch

I now have this function:

def ref_date_epoch():
    ref_date_epoch_result = "{ref_date}000".format(ref_date=int(parse(row[15]).epoch))
    return ref_date_epoch_result

This successfully returns a Unix time object (at least in a format I can use). However, when I reference this function later on in my code, like so (it's towards the bottom as a value for the referral_date key):

        update_company_payload = {
                    "properties": [
                        {"name":"name", "value": row[0] },
                        {"name":"hubspot_owner_id", "value": hubspot_owner_id },
                        {"name":"account_type", "value": row[2] },
                        {"name":"industry", "value": row[3] },
                        {"name":"product", "value": row[4] },
                        {"name":"address", "value": row[5] },
                        {"name":"address2", "value": row[6] },
                        {"name":"city", "value": row[7] },
                        {"name":"state", "value": row[8] },
                        {"name":"zip", "value": row[9] },
                        {"name":"country", "value": row[10] },
                        {"name":"phone", "value": row[11] },
                        {"name":"website", "value": row[12] },
                        {"name":"credit_safe_limit", "value": row[13] },
                        {"name":"credit_safe_rating", "value": row[14] },
                        {"name":"referral_date", "value": ref_date_epoch() },
                        {"name":"referred_from", "value": row[16] },
                        {"name":"referred_to", "value": row[17] },
                        {"name":"source", "value": row[18] }
                      ]
                }

I get this:

Traceback (most recent call last):
  File "wta_goldvision_company_import.py", line 187, in <module>
    run()
  File "wta_goldvision_company_import.py", line 183, in run
    update_company(create_company(), get_owner_id())
  File "wta_goldvision_company_import.py", line 162, in update_company
    {"name":"referral_date", "value": ref_date_epoch() },
  File "wta_goldvision_company_import.py", line 141, in ref_date_epoch
    ref_date_epoch_result = "{ref_date}000".format(ref_date=int(parse(row[15]).epoch))
  File "build/bdist.macosx-10.11-intel/egg/delorean/interface.py", line 68, in parse
  File "build/bdist.macosx-10.11-intel/egg/dateutil/parser.py", line 1161, in parse
  File "build/bdist.macosx-10.11-intel/egg/dateutil/parser.py", line 555, in parse
ValueError: String does not contain a date.

Why does the function work on it's own, but not when its used elsewhere? I've tried just manually entering the date like so:

ref_date_epoch_result = "{ref_date}000".format(ref_date=int(parse("23/12/2015").epoch))

And it works fine. Why would the dateutil parse function have no problems doing its thing outside of the JSON payload, but as soon as it gets referenced inside that thing, it can't seem to interpret the value of row[15] as a string - what am I missing?

Latest Traceback:

23/03/2015
Hermes Transport Logistics GMBH updated successfully.

Traceback (most recent call last):
  File "wta_goldvision_company_import.py", line 187, in <module>
    run()
  File "wta_goldvision_company_import.py", line 183, in run
    update_company(create_company(), get_owner_id())
  File "wta_goldvision_company_import.py", line 162, in update_company
    {"name":"referral_date", "value": ref_date_epoch(row[15]) },
  File "wta_goldvision_company_import.py", line 142, in ref_date_epoch
    return "{ref_date}000".format(ref_date=int(parse(datestr).epoch))
  File "build/bdist.macosx-10.11-intel/egg/delorean/interface.py", line 68, in parse
  File "build/bdist.macosx-10.11-intel/egg/dateutil/parser.py", line 1161, in parse
  File "build/bdist.macosx-10.11-intel/egg/dateutil/parser.py", line 555, in parse
ValueError: String does not contain a date.

This is the entire function that the above resides in (which again is part of a larger whole):

def update_company( company_id, hubspot_owner_id ):

    # Allows HubSpot 15 seconds to reflect the changes made by the create_company() and get_owner_id() functions.
    sleep(15.0)

    # Open the CSV, use commas as delimiters, store it in a list called "data", then find the length of that list.
    with open(os.path.basename(theCSV),"r") as f:
        reader = csv.reader(f, delimiter = ",", quotechar="\"")
        next(reader)
        for row in reader:

            def ref_date_epoch(datestr):
                print(datestr)
                return "{ref_date}000".format(ref_date=int(parse(datestr).epoch))

            # Set up the JSON payload ...
            update_company_payload = {
                        "properties": [
                            {"name":"name", "value": row[0] },
                            {"name":"hubspot_owner_id", "value": hubspot_owner_id },
                            {"name":"account_type", "value": row[2] },
                            {"name":"industry", "value": row[3] },
                            {"name":"product", "value": row[4] },
                            {"name":"address", "value": row[5] },
                            {"name":"address2", "value": row[6] },
                            {"name":"city", "value": row[7] },
                            {"name":"state", "value": row[8] },
                            {"name":"zip", "value": row[9] },
                            {"name":"country", "value": row[10] },
                            {"name":"phone", "value": row[11] },
                            {"name":"website", "value": row[12] },
                            {"name":"credit_safe_limit", "value": row[13] },
                            {"name":"credit_safe_rating", "value": row[14] },
                            {"name":"referral_date", "value": ref_date_epoch(row[15]) },
                            {"name":"referred_from", "value": row[16] },
                            {"name":"referred_to", "value": row[17] },
                            {"name":"source", "value": row[18] }
                          ]
                    }
            #'https://api.hubapi.com/companies/v2/companies/10444744?hapikey=demo'
            update_company_call = "https://api.hubapi.com/companies/v2/companies/{created_company_id}?hapikey={hapikey}".format(hapikey=wta_hubspot_api_key, created_company_id=company_id)
            headers = {'content-type': 'application/json'}
            data = json.dumps(update_company_payload)
            update_company_response = requests.put(update_company_call, data=data, headers=headers)
            #print(json.dumps(update_company_response.json(), sort_keys=True, indent=4))
            #print (update_company_response.status_code)
            if update_company_response.status_code == 200:
                print("{companyName} updated successfully.".format(companyName=row[0]))
            else:
                print("Something went wrong with the {companyName} update.".format(companyName=row[0]))
                print(json.dumps(update_company_response.json(), sort_keys=True, indent=4))
Marko
  • 102
  • 1
  • 10

2 Answers2

0

It seems that row[15] is None on some row. An empty value in the CSV file maybe?

Also, you can define the ref_date_epoch function outside the loop.

Old answer:

Instead of putting row[15] inside the ref_date_epoch function, give it as an argument.

def ref_date_epoch(datestr):
    return "{ref_date}000".format(ref_date=int(parse(datestr).epoch))

Then 

...
{"name":"referral_date", "value": ref_date_epoch(row[15])},
...
Sevanteri
  • 3,749
  • 1
  • 23
  • 27
  • Hey @Sevanteri thanks for responding; I just tried that, and received the same traceback as above. However i tried `print(ref_date_epoch(row[15]))` and it returned `1427068800000` as expected. So weird. Still not working sadly - I can't think of any other ways I could explicitly extract the value of `row[15]` as a string... – Marko Mar 17 '16 at 12:13
  • Did you remember to change the line in the `payload`? – Sevanteri Mar 17 '16 at 12:16
  • And if you get further errors, please post the traceback for them. – Sevanteri Mar 17 '16 at 12:17
  • I did yes, exactly as you've outlined above. Altered the function, and the line in the payload. – Marko Mar 17 '16 at 12:17
  • I'll add them to the initial post above. – Marko Mar 17 '16 at 12:18
  • Right, put `print(datestr)` before the return in the function, to check what you're trying to feed it. – Sevanteri Mar 17 '16 at 12:21
  • Ok i've added `print(datestr)` and as you can see above in the latest traceback, it shows the date `23/03/2015`. I think the key is that the parse function needs the date to be a string, but i've tried interpolating it, or running it through the `str()` function, and nothing seems to want to convert it to a string... – Marko Mar 17 '16 at 12:32
  • 1
    It seems that you are looping through some data and on the second loop your `row[15]` is actually `"Hermes Transport Logistics GMBH updated successfully."` which of course is not a date. :) – Sevanteri Mar 17 '16 at 12:40
  • Haha :) It does look that way from the output above, but the Herme's line is just part of something else. I'll update my original post with the full function that this all resides within. – Marko Mar 17 '16 at 12:46
  • Right, seems so. Well I guess that `row[15]` is `None` on some row. – Sevanteri Mar 17 '16 at 12:50
  • Well, I feel like an idiot - you were right @Sevanteri - I double checked the CSV I was working with - low and behold, the majority of rows below didn't have dates. I spent an embarrassingly long time on this, thanks for your help :) - I'll most my solution (inspired by you) above. – Marko Mar 17 '16 at 13:06
0

The Solution:

Turns out mental fatique makes fools of us all - I was looping through a CSV, and apparently the first row, my test row as it turned out, was the only row for a while which actually contained a date, hence why it crashed when it reached the second row. So I've modified the ref_date_epoch function to be this:

        def ref_date_epoch(datestr):
            if len(datestr) > 0:
                return "{ref_date}000".format(ref_date=int(parse(datestr).epoch))
            else:
                None
Marko
  • 102
  • 1
  • 10