0

I'm a researcher currently based in New Delhi where I'm helping adapt and validate a psychometric questionnaire. For this I need help with fixing the error in calculating ordinal alpha in R using the package psych.

Fixing the Error

The questionnaire is a 7-point Likert-type scale (from 'least likely' to 'most likely') with 30 items and 6 sub-scales. For these sub-scales I'm interested in calculating their ordinal alphas which as far as I understand uses the polychoric correlation matrix rather than the pearson's correlation matrix used in Chronbach's alpha to calculate the alpha scores.

To accomplish this, I'm using R, and specifically the pack psych and the steps outlined in the practical guide, "Estimating ordinal reliability for Likert-type and ordinal item responses data..." (Gadderman et al, 2012). Calculating the ordinal alpha score happens in essentially two commands:

  1. data.pc <- polychoric(data)which creates the polychoric correlation matrix for 'data.pc'
  2. alpha(data.pc$rho) which provides the ordinal alpha score for dataset 'data'

For 5 of my 6 subscales, multiple errors occur in Step 1 -- the creation of the polychoric correlation matrix. I provide two examples below both with 40 responses, but from two different sub-scales. With the first sub-scale with 3 items (ABC) there is a bunch of errors in calculating polychoric(). For the second sub-scale with 3 items (DEF) there is not an error in calculating polychoric and I can subsequently calculate alpha()(which ends up being 0.81 - a good score!). What's goin on?

Data

ABC <- " A B C
1    7    7    7
2    2    7    1
3    2    2    1
4    3    3    1
5    2    3    1
6    3    4    5
7    1    1    1
8    1    1    1
9    4    1    2
10   1    3    1
11   3    1    1
12   5    2    2
13   1    1    1
14   1    1    1
15   1    1    1
16   4    7    1
17   5    1    2
18   4    1    7
19   6    2    1
20   7    7    7
21   3    1    1
22   1    1    1
23   1    1    1
24   1    7    1
25   1    1    1
26   1    1    1
27   1    4    1
28   2    1    1
29   4    7    1
30   7    7    7
31   1    1    7
32   1    1    1
33   1    1    1
34   7    2    2
35   1    2    2
36   6    7    7
37   2    7    5
38   1    2    1
39   1    1    3
40   1    1    1"

ABC <- read.table(text=ABC, header=TRUE) # Dataset with errors

ABC.pc <- polychoric(ABC)
alpha(ABC.pc$rho)


DEF <- " D E F
1    7   7    7
2    7   7    1
3    1   1    1
4    1   1    4
5    1   6    1
6    5   2    2
7    1   3    1
8    1   1    1
9    3   2    3
10   7   4    7
11   2   1    1
12   1   2    1
13   1   1    1
14   4   5    7
15   1   2    2
16   1   1    1
17   4   7    5
18   7   7    1
19   6   6    4
20   7   7    7
21   7   7    1
22   3   3    1
23   4   3    1
24   1   1    1
25   1   1    1
26   1   1    1
27   1   1    4
28   7   7    1
29   4   4    1
30   7   7    4
31   7   2    6
32   1   1    1
33   1   3    1
34   7   2    2
35   2   1    2
36   1   4    4
37   1   1    1
38   1   1    1
39   1   1    5
40   1   1    1"

DEF <- read.table (text=DEF, header= TRUE) #Dataset without errors

DEF.pc <- polychoric(DEF) 
alpha(DEF.pc$rho)

Solutions?

As mentioned in the data entry section (pg.7) of Gadderman et al.'s guide, "the items should have ordinal data with consecutive numbers." This is likely where my issue is, because my respondents often omit one or more values (for example dataset ABC omits values '5' and '6' in item 'B' - one of three that make up the ABC subscale). I have already removed multivariate outliers, but I do expect the responses to have high skew. I cannot collect anymore data either to fill in these gaps.

The only potential solution I've found is in a thread with someone who had a very similar issue. From what I understand, what the statistician (John Fox, Dep. Sociology, McMaster University) suggests is re-ranking responses: For example, if respondents don't respond '5' and '6' then responses with '7' would be re-coded to '5', so that it fits in series with the answers '1,2,3,4'. I realize that the package polychor here is different than the package I'm using, but I think the principle is the same.

My question is: is 're-ranking' in this way appropriate (i.e. will the resulting ordinal alpha scores be accurate estimates)? If not: are there any other solutions?

Community
  • 1
  • 1
Benjamin
  • 1
  • 3
  • Can you please `dput` your data and show your errors? – Hack-R Jun 26 '16 at 16:29
  • I think `ordinal_alpha()` function from the [jugRu](https://rdrr.io/github/jogrue/jogRu/man/ordinal_alpha.html) Package does the trick – Björn Oct 28 '20 at 13:45

1 Answers1

1

Two problems:

  1. the help info seems to think the max # of categories would be 5. It may not be necessary to use this method to estimate reliability for Likert-type items with 7 categories.

  2. There are too many missing values in the data set provided. Even though the data set doesn't contain any NAs, not all of the ordinal categories are represented. It can't generate the estimate you're looking for giving the sparse amount of information provided.

You can edit your data to illustrate the conditions under which this would be expected to function. Example:

# read in data like above
ABC <- read.table(text = ABC, header = TRUE)
# restrict upper range to 5
ABC[ABC > 5] <- 5
# fill in missing values (scale value 4 on variable C)
ABC[1, 3] <- 4

library(psych)
ABC.pc <- polychoric(ABC)
alpha(ABC.pc$rho)

After making those changes (1-5 scale with no missing intervals) those procedures estimated without error for me.

mkearney
  • 1,266
  • 10
  • 18