1

I have set of search documents that have a DateField that I would like to sort by. The values in this field also contain the time. When I try to sort descending by this field, I'm getting the dates to sort correctly, but it seems as though the time is ignored ie:

{
  "results": [
    {
      "photo_create_date": "2016-01-04T16:51:39.096000",
    },
    {
      "photo_create_date": "2016-01-04T17:55:36.483000",
    },
    {
      "photo_create_date": "2016-01-04T22:46:37.141000",
    },
    {
      "photo_create_date": "2016-01-04T16:51:13.450000",
    },
    {
      "photo_create_date": "2016-01-04T22:44:10.289000",
    },
    {
      "photo_create_date": "2016-01-04T22:36:28.252000",
    },
    {
      "photo_create_date": "2015-12-30T18:06:34.511000",
    }
  ]
}

Any idea how to fix this or is this a limitation of the GAE search API?

Brandon
  • 2,886
  • 3
  • 29
  • 44
  • 1
    I'm sure you can store it as Number Field with timestamp as a workaround. I suggest you to fill a bug in their bug tracker about this. – Dmytro Sadovnychyi Jan 05 '16 at 06:16
  • Hi Dmytro, that's actually the solution I built out yesterday. Good idea on filing the bug though. I'll post the ticket here when I get to it later today. Thanks for the response! – Brandon Jan 05 '16 at 16:10
  • https://code.google.com/p/googleappengine/issues/detail?id=12650&thanks=12650&ts=1452021081 – Brandon Jan 05 '16 at 19:11

2 Answers2

1

Seeing as though, this seems to be a bug I had to roll my own solution. Here is what I used:

create_date_aware = pytz.utc.localize(item.create_date)
epoch_datetime = datetime.datetime(1970, 1, 1, tzinfo=pytz.utc)
create_timestamp = (create_date_aware - epoch_datetime).total_seconds()

All you have to do is store this value as a NumberField and it works out pretty well. Some credit goes to this question in figuring out how to do this:

python - datetime with timezone to epoch

Here is the bug I filed with Google:

https://code.google.com/p/googleappengine/issues/detail?id=12650&thanks=12650&ts=1452021081

They rejected this as works as intended, so I created a feature request:

https://code.google.com/p/googleappengine/issues/detail?id=12651&thanks=12651&ts=1452038680

Community
  • 1
  • 1
Brandon
  • 2,886
  • 3
  • 29
  • 44
0

This is the documented behaviour.

While it looks like you have a work around already, you have a few options:

  • Store as a numeric as seconds since epoch
  • Store as a numeric as millis since epoch, however you will need to divide by at least 1000 to be able to fit in the numeric range available in search index
  • Use multiple fields (date, time, possibly timezone)

If you need to retain timezones, you will need to use a multifield approach, with a canonical form in seconds/millis since epoch for comparison.

Nick
  • 1,822
  • 10
  • 9