Find last-gazed-at person in Q-A sequences

Question

I have data on gaze behavior during Question and Answer sequences; gazes are recorded for each speaker A, B, and C in columns A_aoi, B_aoi, and C_aoi, gaze durations are recorded in columns A_aoi_dur, B_aoi_dur, and C_aoi_dur:

df <- data.frame(
  Speaker = c("ID01.A", NA, "ID01.B", "ID33.B", "ID33.A", "ID33.C"),
  Utterance = c("Who did it?", NA, "Peter did.", "So you're coming?", "erm", "Yes, sure."),
  Sequ = c(1,1,1,2,2,2),
  Q = c("q_wh", "", "", "q_decl", "", ""),
  A_aoi = c("C*B*B", "B", "B*", "B*C", "*C", "B*"),
  A_aoi_dur = c("1,2,3,4,5", "1", "1,2", "1,2,3", "1,2", "1,2"),
  B_aoi = c("C*A", "*A", "A*", "A*C", "C", "*C"),
  B_aoi_dur = c("1,2,3", "1,2", "1,2", "1,2,3", "1", "1,2"),
  C_aoi = c("A*A", "A", "A*", "B*C*B", "*B", "B*A"),
  C_aoi_dur = c("1,2,3", "1", "1,2", "1,2,3,4,5", "1,2", "1,2,3")
)

What I need to find out is which person the questioner, upon finishing their question, is gazing at last.

I've been trying to get there with this sequence of operations but have got stuck:

library(tidyr)
library(dplyr)
library(stringr)
df %>%
  # for each `Sequ`...
  group_by(Sequ) %>%
  mutate(
    # Who is the question by?
    Quest_by = sub(".*(.)$", "\\1", first(Speaker)),
    # Who is the answer by?
    Answ_by = sub(".*(.)$", "\\1", last(Speaker))
  ) %>%
  # rename to create column names that are processable by `names_pattern` for `pivot_longer`:
  rename_with(~ str_c(., "_AOI"), ends_with("_aoi")) %>%
  # collect all AOI gazes by A, B, and C into one column: 
  pivot_longer(cols = contains("_aoi"), 
               names_to = c("Gaze_by", ".value"),  # 
               names_pattern = "^(.*)_([^_]+$)"
  ) %>%
  # rename `AOI` and `dur` columns:
  rename(Gaze_to = AOI, Gaze_dur = dur) %>%
  # edit `Gaze_by` and `Gaze_to` values for upcoming analysis:
  mutate(
    # simplify `Gaze_by` values to speaker labels:
    Gaze_by = sub("^(.).*", "\\1", Gaze_by),
    # insert comma into `Gaze_to` as splitting pattern for `separate_rows` command below:
    Gaze_to = str_replace_all(Gaze_to, "(?<=.)(?=.)", ",")
    ) %>%
  # assign each `Gaze_to` and `Gaze_dur` value its own row based on comma as splitting pattern: 
  separate_rows(c(Gaze_to, Gaze_dur), sep = ",", convert = TRUE)

Desired output: (in this or similar form)

  Speaker         Utterance Sequ      Q Q_by  Answ_by  Last_Gaze_to  Last_Gaze_dur
1  ID01.A       Who did it?    1   q_wh    A        B             B              5
2    <NA>              <NA>    1         
3  ID01.B        Peter did.    1         
4  ID33.B So you're coming?    2 q_decl    B        C             C              3
5  ID33.A               erm    2         
6  ID33.C        Yes, sure.    2

EDIT: I've come up with this solution (where df0 is the result of the above operations):

df0 %>%
  filter(Quest_by == Gaze_by) %>%
  group_by(Q, Sequ) %>%
  mutate(Last_Gaze_to = last(Gaze_to),
         Last_Gaze_dur = last(Gaze_dur)) %>%
  ungroup() %>%
  group_by(Line) %>%
  slice_head() %>%
  select(-matches("^G")) %>%
  ungroup() %>%
  mutate(across(c(5:9),
                ~ifelse(Q == "", NA, .)))
# A tibble: 6 × 9
   Line Speaker Utterance          Sequ Q      Quest_by Answ_by Last_Gaze_to Last_Gaze_dur
  <int> <chr>   <chr>             <dbl> <chr>  <chr>    <chr>   <chr>                <int>
1     1 ID01.A  Who did it?           1 q_wh   A        B       B                        5
2     2 NA      NA                    1 NA     NA       NA      NA                      NA
3     3 ID01.B  Peter did.            1 NA     NA       NA      NA                      NA
4     4 ID33.B  So you're coming?     2 q_decl B        C       C                        3
5     5 ID33.A  erm                   2 NA     NA       NA      NA                      NA
6     6 ID33.C  Yes, sure.            2 NA     NA       NA      NA                      NA

Thanks for anybody who did take the trouble of looking into this difficult question. Tips for improvement of the solution are well taken!

I think we need more information about the problem to answer it. On first reading, for example, how do we interpret your `aoi` columns? What's the difference between "C*A", and "A*C"? or betwen "*A", and "A*". Also, you want to know who was looked at *last*, but you given us gaze durations, not start or end times. Do we assume all gazes start at the same instant? And how do we interpret a duration of "1,2,3,4,5"? — Limey, Dec 24 '21 at 13:33
If that fully solves the issue, you might as well post it as an answer — camille, Dec 24 '21 at 15:59

score 2 · Accepted Answer · answered Dec 25 '21 at 11:39

Here is a try. I do not have any experience with 'gazes' etc...

It took me some time and some help (see here Conditionally take value from column1 if the column1 name == first(value) from column2 BY GROUP thanks to @tmfmnk.

I hope this potpourri of code may help. I left the code as it is because of sake of readability. I am sure one can fine tune it. Main parts of what I tried to do is in the blocks.

library(tidyverse)

df %>% 
  # block1
  separate(Speaker, c("SpeakerID", "SpeakerName")) %>% 
  group_by(Sequ) %>% 
  mutate(Q_by = first(SpeakerName)) %>% 
  mutate(Answer_by = last(na.omit(SpeakerName))) %>%
  rename_with(~str_replace(.x, '_aoi', ''), matches('aoi')) %>% 
  # block 2
  rowwise() %>%
  mutate(Last_Gaze_to = get(Q_by)) %>%
  group_by(Sequ, Q_by) %>%
  mutate(Last_Gaze_to = str_extract(last(Last_Gaze_to), '[A-Z]')) %>% 
  rename(A_aoi = A, B_aoi = B, C_aoi = C) %>% 
  rename_with(~str_replace(.x, '_dur', ''), matches('dur')) %>% 
  # block 3
  rowwise() %>% 
  mutate(Last_Gaze_dur = get(Q_by)) %>% 
  group_by(Sequ, Q_by) %>% 
  mutate(Last_Gaze_dur = first(Last_Gaze_dur)) %>% 
  mutate(Last_Gaze_dur = str_sub(Last_Gaze_dur, -1,-1)) %>% 
  # block 4
  ungroup() %>% 
  group_by(Sequ) %>% 
  mutate(across(c(Q_by, Answer_by, Last_Gaze_to, Last_Gaze_dur), ~ifelse(duplicated(.), NA,first(.)))) %>% 
  rename(A_aoi_dur = A, B_aoi_dur=B, C_aoi_dur=C)

Output:

oups:   Sequ [2]
  SpeakerID SpeakerName Utterance          Sequ Q        A_aoi A_aoi_dur B_aoi B_aoi_dur C_aoi C_aoi_dur Q_by  Answer_by Last_Gaze_to Last_Gaze_dur
  <chr>     <chr>       <chr>             <dbl> <chr>    <chr> <chr>     <chr> <chr>     <chr> <chr>     <chr> <chr>     <chr>        <chr>        
1 ID01      A           Who did it?           1 "q_wh"   C*B*B 1,2,3,4,5 C*A   1,2,3     A*A   1,2,3     A     B         B            5            
2 NA        NA          NA                    1 ""       B     1         *A    1,2       A     1         NA    NA        NA           NA           
3 ID01      B           Peter did.            1 ""       B*    1,2       A*    1,2       A*    1,2       NA    NA        NA           NA           
4 ID33      B           So you're coming?     2 "q_decl" B*C   1,2,3     A*C   1,2,3     B*C*B 1,2,3,4,5 B     C         C            3            
5 ID33      A           erm                   2 ""       *C    1,2       C     1         *B    1,2       NA    NA        NA           NA           
6 ID33      C           Yes, sure.            2 ""       B*    1,2       *C    1,2       B*A   1,2,3     NA    NA        NA           NA

Find last-gazed-at person in Q-A sequences

1 Answers1

Linked