1

I want to compute the IRR for a dataset with two raters (ICC2, Two-way-random-effects model). I wanted to use the ICC function from the psych-package but wonder how I need to restructure my data to have one row per participant and one column for each rater.

The dataset looks similar to the sample data below:

data <- data.frame(
  Subject = c(1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 
              3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4), 
  Condition = c("v_1", "v_1.1", "v_1.2", "v_2", "v_2.1", "v_2.2", "v_3", "v_3.1", "v_3.2", "v_1", "v_1.1", "v_1.2", "v_2", "v_2.1", "v_2.2", "v_3", "v_3.1", "v_3.2", "v_1", "v_1.1", "v_1.2", "v_2", "v_2.1", "v_2.2", "v_3", "v_3.1", "v_3.2", "v_1", "v_1.1", "v_1.2", "v_2", "v_2.1", "v_2.2", "v_3", "v_3.1", "v_3.2"), 
  Rater1 = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 9, 8, 7, 6, 5, 4, 3, 2, 1, 1, 2, 3, 4, 5, 6, 7, 8, 9,  9, 8, 7, 6, 5, 4, 3, 2, 1), 
  Rater2 = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 9, 8, 7, 6, 5, 4, 3, 2, 1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 9, 8, 7, 6, 5, 4, 3, 2, 1)
)

I have 4 participants, each variable (3, see column condition) was measured by three items (v_1:v_3.2) by Rater 1 and Rater 2.

I really appreciate your help, thank you very much!

joh_anna
  • 11
  • 2

1 Answers1

0

I guess you want something like this :

library(tidyverse)
data_wider <- pivot_wider(data, names_from = Condition, values_from = c(Rater1, Rater2))
J.Ofoaks
  • 36
  • 3
  • Thank you very much - this definitely helps! However, I am still wondering whether I can transform my data in such a way that I can calculate the ICC for every variable (across the 3 Items). Can you follow my thoughts? I guess I need to make subsets for every variable..?! – joh_anna Oct 10 '22 at 14:26
  • If I understand well, you want to compute the ICC for every V1s conditions together (V_1, V_1.1, V_1.2) and so on for V2s and V3s. By design the psych::ICC function takes only a matrix (or df) of one measure by subject (rows) across some judges (columns). So, my statistical feeling would be that you need to perform a CFA then do your ICC on the single created variables one a the time. – J.Ofoaks Oct 11 '22 at 15:39