ValueError: Unknown protobuf attr type

Question

Getting an error in executing the code. I have a datastore entity which has a property of type Date. An example date property value stored in an entity for a particular row is 2016-01-03 (19:00:00.000) EDT

The code i am executing is filtering the entity values based on date greater than 2016-01-01. Any idea what is wrong with the code

Error

ValueError: Unknown protobuf attr type <type 'datetime.date'>

Code

import pandas as pd
import numpy as np
from datetime import datetime

from google.cloud import datastore
from flask import Flask,Blueprint
app = Flask(__name__)

computation_cron= Blueprint('cron.stock_data_transformation', __name__)
@computation_cron.route('/cron/stock_data_transformation')
def cron():
ds = datastore.Client(project="earningspredictor-173913")
query = ds.query(kind='StockPrice')
query.add_filter('date', '>', datetime.strptime("2016-01-01", '%Y-%m-%d').date())

dataframe_data = []
temp_dict = {}
for q in query.fetch():
    temp_dict["stock_code"] = q["stock_code"]
    temp_dict["date"] = q["date"]
    temp_dict["ex_dividend"] = q["ex_dividend"]
    temp_dict["split_ratio"] = q["split_ratio"]
    temp_dict["adj_open"] = q["adj_open"]
    temp_dict["adj_high"] = q["adj_high"]
    temp_dict["adj_low"] = q["adj_low"]
    temp_dict["adj_close"] = q["adj_close"]
    temp_dict["adj_volume"] = q["adj_volume"]
    dataframe_data.append(temp_dict)
sph = pd.DataFrame(data=dataframe_data,columns=temp_dict.keys())
# print sph.to_string()
query = ds.query(kind='EarningsSurprise')
query.add_filter('act_rpt_date', '>', datetime.strptime("2016-01-01", '%Y-%m-%d').date())

dataframe_data = []
temp_dict = {}
for q in query.fetch():
    temp_dict["stock_code"] = q["stock_code"]
    temp_dict["eps_amount_diff"] = q["eps_amount_diff"]
    temp_dict["eps_actual"] = q["eps_actual"]
    temp_dict["act_rpt_date"] = q["act_rpt_date"]
    temp_dict["act_rpt_code"] = q["act_rpt_code"]
    temp_dict["eps_percent_diff"] = q["eps_percent_diff"]
    dataframe_data.append(temp_dict)
es = pd.DataFrame(data=dataframe_data,columns=temp_dict.keys())

Where's the model definition? – marcadian Aug 17 '17 at 19:35 — marcadian, Aug 17 '17 at 19:35

Dan Cornilescu · Answer 1 · 2017-08-17T21:33:06.203

3

You seem to be using the generic google-cloud-datastore client library, not the NDB Client Library.

For google-cloud-datastore all date and/or time properties have the same format. From Date and time:

JSON

field name: timestampValue

type: string (RFC 3339 formatted, with milliseconds, for instance 2013-05-14T00:01:00.234Z)

Protocol buffer

field name: timestamp_value

type: Timestamp

Sort order: Chronological

Notes: When stored in Cloud Datastore, precise only to microseconds; any additional precision is rounded down.

So when setting/comparing such properties try to use strings formatted as specified (or integers for protobuf Timestamp?), not directly objects from the datetime modules (which work with the NDB library). The same might be true for queries as well.

Note: this is based on documentation only, I didn't use the generic library myself.

edited Aug 17 '17 at 21:33

answered Aug 17 '17 at 21:27

Dan Cornilescu

39,470
12
57
97

How do i build a RFC 3339 formatted timestamp (2016-01-03 (19:00:00.000) EDT) value in python for comparsion to another timestamp value? – user845405 Aug 19 '17 at 17:22
https://stackoverflow.com/questions/8556398/generate-rfc-3339-timestamp-in-python – Dan Cornilescu Aug 19 '17 at 17:27
Getting an error File "/Users/Ram/fintech-api/flex/internal/cron/stock_data_transformation.py", line 3, in from datetime import tzinfo, timedelta, datetime, timezone ImportError: cannot import name timezone – user845405 Aug 19 '17 at 17:44

ValueError: Unknown protobuf attr type

1 Answers1