4

How to count the number of values per column above a sequence of thresholds ?

i.e.: calculate for each column, the number of values above 100, then above 150, then above ... and store the results in a data frame ?

# Reproductible data
# (Original data is daily streamflow values organized in columns per year)

set.seed(1234)
data = data.frame("1915" = runif(365, min = 60, max = 400),
                  "1916" = runif(365, min = 60, max = 400),
                  "1917" = runif(365, min = 60, max = 400))

# my code chunck

mymin = 75
mymax = 400
my step = 25

apply(data, 2, function (x) {
  for(i in seq(mymin,mymax,mystep)) {
    res = (sum(x > i)) # or nrow(data[x > i,])
    return(res)
  }
})

This code works well for one iteration, but I can't store the result of each iteration in a data frame.

I also tried approaches such as :

for (i in 1:n){
  seuil = seq(mymin, mymax, my step)
  lapply(data, function(x) {
    res [[i]] = nrow(data[ x > seuil[i], ])
    return(res)}
})

Which does not work really well...

The output would be something like :

year n value above 75 n values above 100 n value above ...
1915 348 329 ...
1916 351 325 ...
... ... ... ...

Thanks for your comments and suggestions :)

Alex
  • 158
  • 5

2 Answers2

2

You can try :

vals <- seq(mymin,mymax,mystep)
mat <- sapply(vals, function(x) sapply(data, function(y) sum(y > x)))
colnames(mat) <- paste0('values_above_', vals)
mat

#      values_above_75 values_above_100 values_above_125 values_above_150 values_above_175
#X1915             348              329              303              276              235
#X1916             351              325              305              277              252
#X1917             345              315              291              260              236

#      values_above_200 values_above_225 values_above_250 values_above_275 values_above_300
#X1915              212              186              153              126              104
#X1916              226              204              181              146              118
#X1917              208              186              161              133               99

#      values_above_325 values_above_350 values_above_375 values_above_400
#X1915               74               49               28                0
#X1916               92               62               40                0
#X1917               81               60               34                0
Ronak Shah
  • 377,200
  • 20
  • 156
  • 213
0
myseq <- seq(75, 400, by=25)
as.data.frame(do.call(rbind, lapply(data, function(z) table(findInterval(z, myseq)))))
#        0  1  2  3  4  5  6  7  8  9 10 11 12 13
# X1915 17 19 26 27 41 23 26 33 27 22 30 25 21 28
# X1916 14 26 20 28 25 26 22 23 35 28 26 30 22 40
# X1917 20 30 24 31 24 28 22 25 28 34 18 21 26 34

or if you like the factor levels that R will come up with using cut, then

as.data.frame(do.call(rbind, lapply(data, function(z) table(cut(z, myseq)))))
#       (75,100] (100,125] (125,150] (150,175] (175,200] (200,225] (225,250] (250,275] (275,300] (300,325] (325,350] (350,375] (375,400]
# X1915       19        26        27        41        23        26        33        27        22        30        25        21        28
# X1916       26        20        28        25        26        22        23        35        28        26        30        22        40
# X1917       30        24        31        24        28        22        25        28        34        18        21        26        34
r2evans
  • 141,215
  • 6
  • 77
  • 149
  • Thank you! Both codes work very well (the first one gives the following error message ```` Warning Message : In (function (..., deparse.level = 1) : the number of result columns is not a multiple of the vector length (arg 2) ```` with no impact on the results however :) I was not familiar with the `findInterval()` function. Would it be the same if I wanted to retrieve an increasing number of values in increasing intervals? like 0:75, 0:100, 0:125 ... – Alex Feb 10 '21 at 02:53