3

what am I doing wrong??

I use 

    dates<- strptime(dataframe$Measurement.Time,"%d.%m.%Y %H:%M",tz="")

to convert the character strings into dates. This works perfectly on 14780 observations. But in two cases it returns NA.

This is an example where it worked. The excerpts are both from the same resulting data frame.

head(dataframe.with.dates)
        date.time      Measurement.Time mü.mü.VWC.1 øC.Temp.1
1 2000-01-10 00:30:00  10.01.2000 0:30       -0.011      -0.6
2 2000-01-10 01:00:00  10.01.2000 1:00       -0.011      -0.6
3 2000-01-10 01:30:00  10.01.2000 1:30       -0.011      -0.6

This is an excerpt of my resulting data frame showing the two results where it went wrong:

subset(dataframe.with.dates,is.na(dataframe.with.dates$date.time))
          date.time Measurement.Time  mü.mü.VWC   øC.Temp
    9572      <NA>  29.03.2015 2:00      -0.011      -0.6
    9573      <NA>  29.03.2015 2:30      -0.011      -0.6

where "date.time" is in POSIXlt and "Measurement.time" is the original date in character.

I have checked the original .txt file where I got the data from but could not find any difference to the measurements above and below.

Ben Bolker
  • 211,554
  • 25
  • 370
  • 453
Habesha
  • 31
  • 3

1 Answers1

3

Googling "daylight savings time 3/29/2015" shows that this was the date of the spring shift to daylight savings time in Europe, e.g.: https://www.timeanddate.com/news/time/europe-starts-dst-2015.html

Time zones are a nightmare: I would have thought using tz="CET" would do the trick, but it doesn't. strptime(mt,"%d.%m.%Y %H:%M",tz="UTC+01:00") doesn't complain, but it creates the times in UTC. It might be easiest to use tz="GMT" and pretend the times are standard even if they're not ...

I'm sure there are duplicates, but it's easier to answer your question than to find them (I did spend a few minutes trying).

Ben Bolker
  • 211,554
  • 25
  • 370
  • 453
  • Right, this is the hour that does not exist when changing to daylight savings time in spring. However my logger does not account for this. It stays on "winter time" all year long. How can I prevent strptime from switching to daylight savings time? – Habesha Jan 11 '18 at 18:25
  • does using `tz="GMT"` help/get you what you want? – Ben Bolker Jan 11 '18 at 18:45
  • Using `"UTC+01:00"` seems the most appropriate for this use case. You say, "but it creates times in UTC". There isn't a way to create what the OP wants without using a timezone that doesn't have DST. There's no such thing as "winter time" in the spring. Instead, just use the winter time offset from UTC. – Joshua Ulrich Jan 12 '18 at 13:30
  • @BenBolker: `tz="GMT"` does help, it recognizes all timestamps, so does "UTC". However it is not "my" time zone, for my application it is fine @JoshuaUlrich: `"UTC+01:00"` is not recognized by my system. It says `"unknown timezone"`. – Habesha Jan 16 '18 at 16:06
  • 1
    @Habesha: try `"UTC+0100"` (no colon). That's what's suggested in `?strptime`. – Joshua Ulrich Jan 17 '18 at 18:04
  • @JoshuaUlrich, I don't understand why, but the proposed solutions all return `unknown timezone`. However I found that `"Etc/GMT+1"` seems to work. [This is where I found it](https://stackoverflow.com/questions/11927433/timezones-in-r-how-to-avoid-ambiguous-terms-such-as-est). Can someone help me with the meaning of the `Est`? – Habesha Jan 24 '18 at 16:48
  • @Habesha: `"Etc"` is an abbreviation of [et cetera](https://en.wikipedia.org/wiki/Et_cetera). You don't say what operating system you're running R on; how dates/times and timezones are handled are OS-dependent. So it could be that your OS doesn't recognize `"UTC+0100"` as a valid timezone, but it does recognize `"Etc/GMT+1"`. – Joshua Ulrich Jan 24 '18 at 17:35