0

I have data which was collected at different sampling regimes, from every three seconds, up to every hour.

I want to get a subsample to this data at nearest to regular intervals of 10 minutes (also at other time intervals, but I assume it will be easy to change the interval once I have the code right).

I have seen similar posts here (eg:How to subsample a data frame based on a datetime column in R ) but no answers which can allow for buffers or 'nearest to' ideas.

The problems are:

  • The data is not sampled regularly. So I can't ask R to give me all data at every ten minute intervals, but I need 'nearest possible to every ten minutes'.
  • I also want to buffer this so that it doesn't give me intervals of less than 9 minutes.
  • Currently my date_time column is being read as a 'factor':

    class(sample$date_time) [1] "factor"

Sample data:

device_no date_time latitude longitude temperature 1 23/04/2012 15:57:22 -33.2415715 19.4810864 27.7 1 23/04/2012 15:58:08 -33.2415396 19.4810666 27.7 1 23/04/2012 15:58:58 -33.2415963 19.48109 27.2 1 23/04/2012 15:59:46 -33.2415137 19.4810624 27.2 1 23/04/2012 16:00:33 -33.2415019 19.4810885 27 1 23/04/2012 16:01:21 -33.241561 19.4810867 26.3 1 23/04/2012 16:02:10 -33.2415579 19.4810926 26.4 1 23/04/2012 16:02:57 -33.2414687 19.4810465 25.6 1 23/04/2012 16:03:45 -33.2415096 19.4810736 24.6 1 23/04/2012 16:05:20 -33.2415707 19.4810614 24.8 1 23/04/2012 16:06:22 -33.2415188 19.4810708 24.6 1 23/04/2012 16:07:12 -33.2415754 19.4810538 24.6 1 23/04/2012 16:08:00 -33.2415054 19.4810874 24.2 1 23/04/2012 16:08:48 -33.2414794 19.4810908 24.3 1 23/04/2012 16:09:36 -33.2415538 19.4810802 24 1 23/04/2012 16:10:25 -33.2413946 19.4811353 23.4 1 23/04/2012 16:11:14 -33.2414529 19.4811084 23.7 1 23/04/2012 16:12:01 -33.2413949 19.4810978 23.5 1 24/04/2012 05:26:39 -33.2415605 19.4810589 23.1 1 24/04/2012 05:56:51 -33.2414826 19.4811049 23 1 24/04/2012 06:01:22 -33.2415975 19.4810535 23 1 24/04/2012 06:02:09 -33.2384224 19.482825 21.7 1 24/04/2012 06:02:58 -33.2380158 19.4833283 20.6 1 24/04/2012 06:03:45 -33.241538 19.4810951 19.9 1 24/04/2012 06:04:34 -33.2416337 19.4810291 19.3 1 24/04/2012 06:05:22 -33.2410841 19.4819002 19.9 1 24/04/2012 06:06:11 -33.2401989 19.4817907 19.9 1 24/04/2012 06:06:57 -33.241593 19.4810426 19.4 1 24/04/2012 06:07:46 -33.241575 19.4810011 18.6 1 24/04/2012 06:08:34 -33.2415497 19.4810493 18.8 1 24/04/2012 06:09:22 -33.2415104 19.4810216 18.4 1 24/04/2012 06:10:11 -33.2416627 19.4810065 18 1 24/04/2012 06:10:59 -33.2414893 19.4811548 18.2 1 24/04/2012 06:11:44 -33.2420604 19.4810295 18.7 1 24/04/2012 06:12:33 -33.2408584 19.4803685 35.5 1 24/04/2012 06:13:20 -33.2407331 19.4805824 38.3 1 24/04/2012 06:25:58 -33.2411718 19.4810405 39.9 1 24/04/2012 06:26:49 -33.2415396 19.4810794 41.4 1 24/04/2012 06:27:56 -33.2415588 19.481089 40 1 24/04/2012 06:28:54 -33.2415257 19.4810381 41 1 24/04/2012 06:29:42 -33.239857 19.4807259 37.6 1 24/04/2012 06:30:29 -33.2409401 19.480927 36.4

Sample outcome:

device_no date_time latitude longitude temperature 1 23/04/2012 15:57:22 -33.2415715 19.4810864 27.7 1 23/04/2012 16:07:12 -33.2415754 19.4810538 24.6 1 24/04/2012 05:26:39 -33.2415605 19.4810589 23.1 1 24/04/2012 05:56:51 -33.2414826 19.4811049 23 1 24/04/2012 06:06:57 -33.241593 19.4810426 19.4 1 24/04/2012 06:25:58 -33.2411718 19.4810405 39.9`

Sorry, my screen shot won't post and this data doesn't appear as a table...

Community
  • 1
  • 1
user3690418
  • 3
  • 3

1 Answers1

0
# Create some data

# start at Jan 1, 2014
start=as.POSIXct("1-Jan-2014 00:00",format="%d-%b-%Y %H:%M")

# end at Jan 31, 2014
end=as.POSIXct("31-Jan-2014 23:00",format="%d-%b-%Y %H:%M")

# sequence the time from start to end and create a continuous time 
# set with observations every 5 minutes with a sd of 1 minute
time=seq.POSIXt(start,end,by=rnorm(1,1*60*5,60))

# create a loop

# you want the buffer for time > 9 minutes
time.diff=9
new.time=time[1]
# seq 10 minute intervals (this is the data we are going to try and match)
time.10min=seq.POSIXt(start,end,by=1*60*10)

# skip the first observation
for(i in 2:length(time.10min)){
  # Look for where the minimum difference occurs in your original dataset
  obs=time[which(min(abs(time.10min[i]-time))==abs(time.10min[i]-time))]

  # see if it meets you buffer requirements. If it doesn't meet your buffer requirements
  # it will take the next observation in your dataset
  if(obs-new.time[i-1]<time.diff){
    obs=time[which(min(abs(time.10min[i]-time))==abs(time.10min[i]-time))+1]
    new.time[length(new.time)+1]=obs
  } else {
    new.time[length(new.time)+1]=obs
  }
}

head(new.time)


# Good luck and hopefully this helps.
s_scolary
  • 1,361
  • 10
  • 21