Determining the duration of an exceedance

Question

I am analysing SCADA data using R.

The problem I need to solve is to analyse a SCADA feed and determine how often the measurements exceeded a certain limit for more than 15 minutes.

The only way I could solve this is by using a for loop, which will make the process very slow because real life application will have thousands of points.

Any suggestions?

Simple example:

set.seed(666)
upper_limit =1.5
sims <- 50
turb <- abs(rnorm(sims))
time <- seq.POSIXt(as.POSIXct(Sys.Date()-1), by=30, length.out=sims)
plot(time,turb, type="l")
abline(h=upper_limit, col="red", lwd=2)

See: http://rpubs.com/pprevos/scada

The answer for this example is: 8 exceedances and I also need to know the duration of each of these.

Answer 1

If your time series was a 1-minute time serie (ie: a time series with 1 minute period), it is easy to get the lengths of intervals exceeding some threshold using rle:

 xx = rle(turb >1.5)
 sum(xx$values==TRUE & xx$lengths >=15)

So here in order to get this time serie, one solution is to approximate it to create a new time serie with more accuracy.

library(xts)
xx = xts(turb,time)
yy = na.approx(merge(xts(,seq.POSIXt(min(time),max(time),by=1)),
      xx))
## optional plot the new and the old time series
plot(x = yy, xlab = "Time",  minor.ticks = FALSE, col = "red")
points(x = xx, col = "darkgreen",pch=20)

Then I compute the number of intervals as explained above :

xx = rle(as.vector(coredata(yy>1.5)))
sum(xx$values==TRUE & xx$lengths >=15)
[1] 6

Note: here I found only 6 intervals..