Parse datetime with timezone using Boost.Date_Time?

Question

I want to parse date time with time-zone using Boost Date Time IO library.

#include <boost/date_time.hpp>
#include <ctime>
#include <sstream>

using namespace boost::gregorian;
using namespace boost::posix_time;

std::chrono::system_clock::time_point ParseDate(const std::wstring& dateText, const wchar_t* const format) {
    ptime time;
    std::wstringstream buffer(dateText);
    buffer.imbue(std::locale(std::locale::classic(), new wtime_input_facet(format)));
    buffer >> time;
    auto timeInfo = to_tm(time);
    auto result = std::chrono::system_clock::from_time_t(std::mktime(&timeInfo));
    return result;
}

TEST_CLASS(DateUtilsTest) {
public:
    TEST_METHOD(ShouldParseUtcDate) {
        auto timePoint = ParseDate(L"2016-12-03T07:09:01-05:00", L"%Y-%m-%dT%H:%M:%S%Q");
        auto time = std::chrono::system_clock::to_time_t(timePoint);
        auto timePoint2 = ParseDate(L"2016-12-03T07:09:01", L"%Y-%m-%dT%H:%M:%S");
        auto time2 = std::chrono::system_clock::to_time_t(timePoint2);
        Assert::IsTrue(time != time2);
    }
}

Here's an online example: https://wandbox.org/permlink/9GEhah5l4uzhgDta

The above test failed because time == time2.

It seems that the time-zone portion doesn't have any impact to the parsing result.

Do you know how to parse a date time string with time-zone using Boost?

The real case is string like "2017-12-21T10:47:58.299Z" (ISO 8601 format, generated by JavaScript: (new Date()).toISOString()), but I didn't find any doc for this format, any ideas?

Environment:

Boost-1.65.1
System is Windows 10
System time-zone is GMT+8
Visual Studio 2015U3

Looks like this indeed just a typo. Then this question can be closed. — , Dec 21 '17 at 14:52
Sure would be easy using [Howard's timezone lib](https://howardhinnant.github.io/date/tz.html) :-) — Howard Hinnant, Dec 21 '17 at 15:37
[For example](https://wandbox.org/permlink/T0Ng1gES8X0Na4uG). — Howard Hinnant, Dec 21 '17 at 15:48
@HowardHinnant Is there a way to parse time like "2017-12-21T10:47:58.299Z"? Is there any doc for format "%Ez"? — user1633272, Dec 21 '17 at 17:26
Yes, you can parse fractional seconds. Just create a type that holds the precision you want to parse into and use that in place of where my example uses `sys_seconds`. For example I could have parsed into a `system_clock::time_point` (which has microseconds precision on my platform), and that would allow for parsing up to microseconds precision with the %S and %T flags. Here is full documentation for all of the parsing flags: ttps://howardhinnant.github.io/date/date.html#from_stream_formatting You can parse your case with `"%FT%TZ"` into a `time_point` with precision milliseconds or greater. — Howard Hinnant, Dec 21 '17 at 17:46
Updated docs link: https://howardhinnant.github.io/date/date.html#from_stream_formatting — Howard Hinnant, Dec 21 '17 at 17:57
Look good. What does "%FT" stands for? I failed with "%Y-%m-%dT%H:%M:%S%TZ" which I thought the same as "%FT%TZ". It's better to document the special %FT, %TZ although it is already complicated. — user1633272, Dec 21 '17 at 18:02
`"%FT"` is equivalent to `"%Y-%m-%dT"`. You can parse with `"%Y-%m-%dT%H:%M:%SZ"`, which is equivalent to `"%FT%TZ"`. — Howard Hinnant, Dec 21 '17 at 18:04
@HowardHinnant https://wandbox.org/permlink/MgdMwlKHnFNxEJb2 I may be wrong, does 'Z' suffix stands for UTC? Why I got same output? — user1633272, Dec 21 '17 at 18:14
`Z` just stands for the literal `Z`. It doesn't mean anything at all. The type `system_clock::time_point` means UTC (excluding leap seconds). If you've got a `system_clock::time_point`, it represents UTC. There are ways to represent local time. You'll need the "tz.h" portion of this library to handle local time / time zones, which requires a source file and some installation on Windows: https://howardhinnant.github.io/date/tz.html#Installation — Howard Hinnant, Dec 21 '17 at 18:18
Slight correction. If the only way you need to represent local time is by parsing UTC offsets as you showed earlier, you can stick with the header-only "date.h". "tz.h" is needed for named IANA time zones, and your computer's current local time zone. — Howard Hinnant, Dec 21 '17 at 18:20
https://en.wikipedia.org/wiki/ISO_8601, there is a 'UTC' section, says: "Z is the zone designator for the zero UTC offset. " F.Y.I — user1633272, Dec 21 '17 at 18:25
If you put a `"Z"` in the format string, the parse will demand to parse a `Z` in the input, else it won't. If you successfully parse into a `time_point`, that quantity will be interpreted as [Unix Time](https://en.wikipedia.org/wiki/Unix_time). You can parse into a `time_point` with or without using a `Z`, your choice. — Howard Hinnant, Dec 21 '17 at 18:30

sehe · Accepted Answer · 2017-12-22T00:01:40.160

I've looked at it long and hard. It seems you're almost completely out of luck:

So you can try making it work with %ZP.

Doing The Heroics

I did the heroics, only to find out that the support for wtime_zone and friends is ... incomplete in the library.

Here it is in all it g(l)ory:

Live On Coliru

#include <boost/date_time.hpp>
#include <boost/date_time/local_time/local_time_io.hpp>
#include <boost/date_time/local_time/local_time.hpp>
#include <boost/date_time/time_zone_base.hpp>
#include <ctime>
#include <chrono>
#include <sstream>

namespace DT = boost::date_time;
namespace LT = boost::local_time;
namespace PT = boost::posix_time;

template <typename CharT = wchar_t> struct TypeDefs {
    using ptime   = PT::ptime;
    using tz_base = DT::time_zone_base<ptime, CharT>;
    using tz_ptr  = boost::shared_ptr<DT::time_zone_base<PT::ptime, CharT> >;
    using ptz_t   = LT::posix_time_zone_base<CharT>;
    using ldt_t   = LT::local_date_time_base<ptime, tz_base>;
};

namespace boost { namespace local_time { 
  //! input operator for local_date_time
  template <class CharT, class Traits, typename Defs = TypeDefs<CharT>, typename local_date_time = typename Defs::ldt_t>
  inline
  std::basic_istream<CharT, Traits>&
  operator>>(std::basic_istream<CharT, Traits>& is, local_date_time& ldt)
  {
    using time_zone_ptr = typename Defs::tz_ptr;
    using posix_time_zone = typename Defs::ptz_t;
    boost::io::ios_flags_saver iflags(is);
    typename std::basic_istream<CharT, Traits>::sentry strm_sentry(is, false);
    if (strm_sentry) {
      try {
        typedef typename local_date_time::utc_time_type utc_time_type;
        typedef typename date_time::time_input_facet<utc_time_type, CharT> time_input_facet;

        // intermediate objects
        std::basic_string<CharT> tz_str;
        utc_time_type pt(DT::not_a_date_time); 

        std::istreambuf_iterator<CharT,Traits> sit(is), str_end;
        if(std::has_facet<time_input_facet>(is.getloc())) {
          std::use_facet<time_input_facet>(is.getloc()).get_local_time(sit, str_end, is, pt, tz_str);
        }
        else {
          time_input_facet* f = new time_input_facet();
          std::locale l = std::locale(is.getloc(), f);
          is.imbue(l);
          f->get_local_time(sit, str_end, is, pt, tz_str);
        }
        if(tz_str.empty()) {
          time_zone_ptr null_ptr;
          // a null time_zone_ptr creates a local_date_time that is UTC
          ldt = local_date_time(pt, null_ptr);
        }
        else {
          time_zone_ptr tz_ptr(new posix_time_zone(tz_str));
          // the "date & time" constructor expects the time label to *not* be utc.
          // a posix_tz_string also expects the time label to *not* be utc.
          ldt = local_date_time(pt.date(), pt.time_of_day(), tz_ptr, local_date_time::EXCEPTION_ON_ERROR);
        }
      }
      catch(...) {
        // mask tells us what exceptions are turned on
        std::ios_base::iostate exception_mask = is.exceptions();
        // if the user wants exceptions on failbit, we'll rethrow our 
        // date_time exception & set the failbit
        if(std::ios_base::failbit & exception_mask) {
          try { is.setstate(std::ios_base::failbit); }
          catch(std::ios_base::failure&) {} // ignore this one
          throw; // rethrow original exception
        }
        else {
          // if the user want's to fail quietly, we simply set the failbit
          is.setstate(std::ios_base::failbit);
        }

      }
    }
    return is;
  }
} }

template <typename CharT = wchar_t> struct DateUtilsBase : TypeDefs<CharT> {

    using base = TypeDefs<CharT>;
    using typename base::ldt_t;
    using typename base::tz_ptr;
    using typename base::ptime;

    static std::tm to_tm(ldt_t const& lt) {
        std::tm v = PT::to_tm(lt.local_time());
        v.tm_isdst = lt.is_dst()? 1:0;
        return v;
    }

    static tz_ptr s_GMT;

    static std::chrono::system_clock::time_point Parse(const std::basic_string<CharT>& dateText, const CharT* const format) {

        ldt_t value(LT::special_values::not_a_date_time, s_GMT);

        std::basic_istringstream<CharT> buffer(dateText);
        buffer.imbue(std::locale(std::locale::classic(), new DT::time_input_facet<ptime, CharT>(format)));

        std::basic_string<CharT> dummy;
        if (buffer >> value && (buffer >> dummy).eof()) {
            std::cout << "DEBUG: " << value.utc_time() << " EOF:" << buffer.eof() << "\n";
            auto timeInfo = PT::to_tm(value.utc_time());
            return std::chrono::system_clock::from_time_t(std::mktime(&timeInfo));
        } else {
            return std::chrono::system_clock::time_point::min();
        }
    }
};

template <> typename DateUtilsBase<wchar_t>::tz_ptr DateUtilsBase<wchar_t>::s_GMT { new ptz_t(L"GMT") } ;
template <> typename DateUtilsBase<char>::tz_ptr    DateUtilsBase<char>::s_GMT    { new ptz_t("GMT")  } ;

#if 1
    using DateUtils = DateUtilsBase<wchar_t>;
    #define T(lit) L##lit
#else
    using DateUtils = DateUtilsBase<char>;
    #define T(lit) lit
#endif

int main() {
    using namespace std::chrono_literals;
    using C = std::chrono::system_clock;
    std::cout << std::boolalpha << std::unitbuf;

    C::time_point with_zone, without_zone;

    // all three equivalent:
    with_zone = DateUtils::Parse(T("2016-12-03T07:09:01 PST-05:00"), T("%Y-%m-%dT%H:%M:%S%ZP"));
    with_zone = DateUtils::Parse(T("2016-12-03T07:09:01 -05:00"), T("%Y-%m-%dT%H:%M:%S%ZP"));
    with_zone = DateUtils::Parse(T("2016-12-03T07:09:01-05:00"), T("%Y-%m-%dT%H:%M:%S%ZP"));

    without_zone = DateUtils::Parse(T("2016-12-03T07:09:01"), T("%Y-%m-%dT%H:%M:%S"));
    std::cout << "time_point equal? " << (with_zone == without_zone) << "\n";

    {
        std::time_t t_with_zone    = C::to_time_t(with_zone);
        std::time_t t_without_zone = C::to_time_t(without_zone);

        std::cout << "time_t equal? " << (t_with_zone == t_without_zone) << "\n";
    }

    std::cout << (without_zone - with_zone) / 1h << " hours difference\n";
}

Yep. That's a bit of a monstrosity. It prints:

DEBUG: 2016-Dec-03 12:09:01 EOF:true
DEBUG: 2016-Dec-03 12:09:01 EOF:true
DEBUG: 2016-Dec-03 12:09:01 EOF:true
DEBUG: 2016-Dec-03 07:09:01 EOF:true
time_point equal? false
time_t equal? false
-5 hours difference

Back To Sanity

In fact, the library authors (wisely) decided that even though streams would be wide or narrow, the local_date_time (or really, just the strings in their time-zone representations) need not be. This is why the library supplied operator>> only supports local_date_time and employs the internal helper function convert_string_type to coerce to narrow-char timezone info:

time_zone_ptr tz_ptr(new posix_time_zone(date_time::convert_string_type<CharT,char>(tz_str)));

With that in mind let's remove a lot "generic-y" cruft. What remains is the addition of error-handling:

if (buffer >> value && (buffer >> dummy).eof()) {
    //std::cout << "DEBUG: " << value.utc_time() << " EOF:" << buffer.eof() << "\n";
    auto timeInfo = boost::posix_time::to_tm(value.utc_time());
    return std::chrono::system_clock::from_time_t(std::mktime(&timeInfo));
} else {
    return std::chrono::system_clock::time_point::min();
}

Live On Coliru

#include <boost/date_time.hpp>
#include <boost/date_time/local_time/local_time_io.hpp>
#include <ctime>
#include <chrono>
#include <sstream>

struct DateUtils {
    using ptime           = boost::posix_time::ptime;
    using time_zone_ptr   = boost::local_time::time_zone_ptr;
    using local_date_time = boost::local_time::local_date_time;

    template <typename CharT>
    static std::chrono::system_clock::time_point Parse(const std::basic_string<CharT>& dateText, const CharT* const format) {
        static time_zone_ptr s_GMT(new boost::local_time::posix_time_zone("GMT"));

        local_date_time value(boost::local_time::special_values::not_a_date_time, s_GMT);

        std::basic_istringstream<CharT> buffer(dateText);
        buffer.imbue(std::locale(std::locale::classic(), new boost::date_time::time_input_facet<ptime, CharT>(format)));

        std::basic_string<CharT> dummy;
        if (buffer >> value && (buffer >> dummy).eof()) {
            //std::cout << "DEBUG: " << value.utc_time() << " EOF:" << buffer.eof() << "\n";
            auto timeInfo = boost::posix_time::to_tm(value.utc_time());
            return std::chrono::system_clock::from_time_t(std::mktime(&timeInfo));
        } else {
            return std::chrono::system_clock::time_point::min();
        }
    }
};

#if 1
    using CharT = wchar_t;
    #define T(lit) L##lit
#else
    using CharT = char;
    #define T(lit) lit
#endif

int main() {
    using namespace std::chrono_literals;
    using C = std::chrono::system_clock;
    std::cout << std::boolalpha << std::unitbuf;

    C::time_point with_zone, without_zone;

    // all three equivalent:
    with_zone = DateUtils::Parse<CharT>(T("2016-12-03T07:09:01 PST-05:00"), T("%Y-%m-%dT%H:%M:%S%ZP"));
    with_zone = DateUtils::Parse<CharT>(T("2016-12-03T07:09:01 -05:00"), T("%Y-%m-%dT%H:%M:%S%ZP"));
    with_zone = DateUtils::Parse<CharT>(T("2016-12-03T07:09:01-05:00"), T("%Y-%m-%dT%H:%M:%S%ZP"));

    without_zone = DateUtils::Parse<CharT>(T("2016-12-03T07:09:01"), T("%Y-%m-%dT%H:%M:%S"));
    std::cout << "time_point equal? " << (with_zone == without_zone) << "\n";

    {
        std::time_t t_with_zone    = C::to_time_t(with_zone);
        std::time_t t_without_zone = C::to_time_t(without_zone);

        std::cout << "time_t equal? " << (t_with_zone == t_without_zone) << "\n";
    }

    std::cout << (without_zone - with_zone) / 1h << " hours difference\n";
}

Whew. From 151 LoC down to 64 LoC. Better

Prints:

time_point equal? false
time_t equal? false
-5 hours difference

Summary:

read (all) the notes in the docs
use %ZP as the only supported input format
use local_date_time because that format string is ignored with ptime (it says so in the notes)
use error handling to make sure no "unparsed" things are left behind in the input

That's a typo, but still doesn't work after fix. Please have a look the updated version. — user1633272, Dec 21 '17 at 14:54
I figured things out the hard way. Boost DateTime is never as user-friendly as one would hope. Anyways... it works now :) Just for completeness I'm going to point out I usually roll with `strptime` and my [`adaptive_parser` helper class](https://stackoverflow.com/questions/44083239/using-boost-parse-datetime-string-with-single-digit-hour-format/44091087#44091087) — sehe, Dec 21 '17 at 22:38

Parse datetime with timezone using Boost.Date_Time?

1 Answers1

Doing The Heroics

Back To Sanity

Summary: