How do I change my panel data from wide to long and also, how do I create a time variable and add it

Question

I am having some trouble getting my code to work. I earlier asked a question on this website that did not solve my problem entirely. "Reshaping the HRS data from wide to long and creating a time variable"

This time I tried to be very clear and precise in describing my data. It looks like something like this, where all the variables start with "r" followed by a number from 1 to 10 followed by the variable measured. The only variable that does not start with "r" is the id-tracker which is called "idhhpn".

This is a sample of how my data is structured, but not exactly my data. My data file is very large and I can't post it here:

df <- structure(list(data = structure(1:4, .Label = c("Ind_1", "Ind_2", 
    "Ind_3", "Ind_4"), class = "factor"), r1weight = c(56, 76, 87, 64
    ),r10weight = c(57, 75, 88, 66), r1height = c(186, 176, 187, 165), r10height = c(187L, 
    173L, 185L, NA), r1bmi = c(23L, 22L, 25L, 21L), r10bmi = c(24L, 23L, 
    29L, 23), r1logass = c(8L, 4L, NA, 2L), r10logass = c(7, 5L, 2, 
    4L), r1vigact = c(1, 0, 1, 1), r10vigact = c(0,0,0,1), idhhpn = c(1,2,3,4), rmale = c(0,0,1,0), rhighs = c(1,1,1,0), rcoll = c(1,0,1,0) ), class = 
    "data.frame", row.names = c(NA, 
    -4L)) 



data       r1weight r10weight r1height r10height r1bmi r10bmi r1logass r10logass r1vigact r10vigact idhhpn rmale rhighs rcoll
1 Ind_1       56        57      186       187    23     24        8         7        1         0      1     0      1     1
2 Ind_2       76        75      176       173    22     23        4         5        0         0      2     0      1     0
3 Ind_3       87        88      187       185    25     29       NA         2        1         0      3     1      1     1
4 Ind_4       64        66      165        NA    21     23        2         4        1         1      4     0      0     0
`

I have 23 variables all observed 10 times (one every year for 10 years). I also have Several dummies like rmale, rhispanic, rblack, rHS, rGED, rCollege and so on.

I wish to convert this into this:

dflong <- structure(list(time = structure(1:12, .Label = c("1", "...","10","1", "...","10","1", "...","10", "1", "...","10"), 
     class = "factor"), idhhpn = c(1,1,1,2,2,2,3,3,3,4,4,4), W = c(56,"...", 57,76,"...",75,87,"...",88,64,"...",66),
     H = c(186,"...",187,176,"...",173,187,"...",185,165,"...","..."), BMI = c(23,"...",24,22,"...",23,25,"...",29,21,"...",23),
     logA = c(8,"...",7,4,"...",5,"...","...",2,2,"...",4), vigact = c(1,"...",0,0,"...",0,1,"...",0,1,"...",1),
     rmale = c(0,"...",0,0,"...",0,1,"...",1,0,"...",0), rhighs = c(1,"...",1, 1,"...",1,1, "...",1,0,"...",0),
     rcoll = c(1,"...",1,0,"...",0,1,"...",1,0,"...",0)),
     class = "data.frame", row.names = c(NA, -12L))`

   time idhhpn W   H BMI logA vigact rmale rhighs rcoll
1     1      1  56 186  23    8      1     0      1     1
2   ...      1 ... ... ...  ...    ...   ...    ...   ...
3    10      1  57 187  24    7      0     0      1     1
4     1      2  76 176  22    4      0     0      1     0
5   ...      2 ... ... ...  ...    ...   ...    ...   ...
6    10      2  75 173  23    5      0     0      1     0
7     1      3  87 187  25  ...      1     1      1     1
8   ...      3 ... ... ...  ...    ...   ...    ...   ...
9    10      3  88 185  29    2      0     1      1     1
10    1      4  64 165  21    2      1     0      0     0
11  ...      4 ... ... ...  ...    ...   ...    ...   ...
12   10      4  66 ...  23    4      1     0      0     0

Where there is also a time variable going from 1 to 10 for each individual for each variable as shown.

Where I have omitted timestamps 2-9 (for readability)

I currently have the following code which I am certain is almost correct.

HRSdata_melt <- HRSdata %>% gather(time,ind,-HRSdata) %>%
                    mutate(time=gsub("r([1-10])", "\\1_",time)) %>%
                    separate(time, into = c("time", "idhhpn")) %>% 
                    spread(idhhpn, ind)

but it gives me the following error which I think is due to some minor error.

Here is an example of dput(head(HRSdata))

hi, can you expain what `HRSdata` is and how it relates to `df`? — Cettt, Mar 25 '19 at 12:56
df means "dataframe" and can be virtually every data set you can imagine. For my specific project I am working with the HRS longitudinal data set (google it if you have to). "HRSdata" is simply the name of my dateframe (df). — Emil Krabbe, Mar 25 '19 at 13:09
yeah, but what does it look like. Can you post the output of `dput(head(HRSdata))`? — Cettt, Mar 25 '19 at 13:10

A. Suliman · Answer 1 · 2019-03-26T04:02:45.507

We can use dplyr::matches to gather over all columns with r followed by a digit or more followed by anything i.e. r1weight, r2weight, ... , r10weight etc. Then mutate and spread

library(dplyr)
library(tidyr)
df %>% gather(key,val,matches('r\\d+.*')) %>% 
       mutate(time=gsub('r(\\d+).*','\\1',key), key=sub('(r\\d+)(.*)','\\2',key)) %>%
       spread(key,val)

r(\\d+).* get one or more digits after r as the first group and return that using \\1

(r\\d+)(.*) get anything after (r\\d+) as the second group and return that using \\2

How do I change my panel data from wide to long and also, how do I create a time variable and add it

1 Answers1