I have validated a criminal justice risk assessment tool designed to predict prison misconduct in a six month follow up period post-assessment (0 = No Misconduct; 1 = Misconduct) using AUCs and predicted probabilities/ORs derived from a logistic regression (Misconduct ~ Risk Assessment Scored Custody Level + Other Predictors).
However, I would like to figure out how to revise the risk assessment tool going forward such that the tool appropriately up-weights tool items which actually predict misconduct and down-weights or eliminates altogether tool items which are not predictive because, as is, the current risk assessment tool contains items which do not predict misconduct risk at both the bivariate and multivariate level. For reference, the risk assessment tool includes 10 different scored items, ranging from history of institutional adjustment/violence (0 = none; 3 = violent incident w/o weapon, 6 = violent incident with weapon) to age (2 = 21 and under; -2 = 45 and above).
If anyone has any recommendations on how to revise the tool going forward which are psychometrically sound, I would appreciate this.
I initially thought I would use the results of the logistic regression predicting misconduct as a function of each of the ten scores on the tool for item-reduction (e.g., exclude variables which are not predictive at the .10 level). However, I recognize that the tool's initial weights are arbitrary and thus, I don't know that relying on a logistic regression using the prior tool's scores is useful.
Because of this, I thought that I should convert the items in the assessment into factor variables (e.g., history of institutional adjustment/violence) and then re-run the regression. However, it is unclear to me how to get from the point of running the logistic model to developing the final scores. Any advice here on either of these questions (1 - do I use the logit model with scores or factor transformations of underlying scores? or 2 - how do I go from logit coefficients to generate scores?) is appreciated.
