0

I downloaded some tweets using 'rtweet' library. Its search_tweets() function creates a list (type) object, while its class is "tbl_df" "tbl" "data.frame". To further work on it, I need to convert this search_tweets() output into a dataframe. 

comments <- search_tweets(
queryString, include_rts = FALSE,
n = 18000, type = "recent",
retryonratelimit = FALSE)

typeof(comments)

list

class(comments)

"tbl_df" "tbl" "data.frame"

I tried to convert list into dataframe by using as.data.frame(), that didn't change the type, I also tried wrapping it into as.dataframe(matrix(unlist(comments))), that didn't change the type as well

commentData <- data.frame(comments[,1]) 
    for (column in c(2:ncol(comments))){
        commentData <- cbind(commentData, comments[,column])
    }
type(comments)

output : list

comments <- as.data.frame(comments)

output : list

Both these codes didn't change the type, but the class. How should I change the type? As, I'd like to store these tweets into a dataframe and consequently write them as csv (write_csv).

As I write the 'comments' to csv, it throws an error. 

write_csv(comments, "comments.csv", append =  TRUE)

Error: Error in stream_delim_(df, path, ..., bom = bom, quote_escape = quote_escape) : Don't know how to handle vector of type list.

dput(comments)

dput(comments) structure(list(user_id = c("1213537010930970624", "770697053538091008", "39194086", "887369171603931137", "924786826870587392", "110154561", "110154561", "1110623370389782528", "1201410499788689408", "1208038347735805953", "15608380", "54892886", "389914405", "432597210", "1196039261125918720" ), status_id = c("1217424480366026753", "1217197024405143552", "1217057752918392832", "1217022975108616193", "1217002616757997568", "1216987196714094592", "1216986705170923520", "1216978052472688640", "1216947780129710080", "1216943924796739585", "1216925375789330432", "1216925016605880320", "1216924608944734208", "1216921598294249472", "1214991714688987136"), created_at = structure(c(1579091589, 1579037359, 1579004154, 1578995863, 1578991009, 1578987332, 1578987215, 1578985152, 1578977935, 1578977016, 1578972593, 1578972507, 1578972410, 1578971693, 1578511572), class = c("POSIXct", "POSIXt"), tzone = "UTC"), screen_name = c("SufferMario", "_Mohammadtausif", "avi_rules16", "Deb05810220", "SriPappumaharaj", "Poison435", "Poison435", "RajeshK38457619", "KK77979342", "beingskysharma", "tetisheri", "sohinichat", "nehadixit123", "panwarsudhir1", "NisarMewati1" ), 

desired output in csv

screengrab of csv format

Shreya Agarwal
  • 676
  • 2
  • 8
  • 20
  • 1
    A data frame actually is a type of list; that might be throwing off your understanding of what you're seeing – camille Jan 16 '20 at 16:46
  • Hi Camille, I just updated my question. with the output I got when I tried to write my 'comments' object as csv. That's what causing the trouble. I think it is still considering it as a list object and not as a dataframe for. the purpose of write_csv() – Shreya Agarwal Jan 16 '20 at 17:04

1 Answers1

1

You don't need to do anything. comments is already a data.frame. It just happens to be a special type of data.frame known as a tibble. But you can use them interchangeably. What do you want to do with comments that you currently cannot? It already should do anything a data.frame can do.

The output from typeof() is rarely helpful as it only shows you how the object is stored, not what it is. Use class() to understand how an object behaves. Nearly all "complex" objects in R are stored as lists.

MrFlick
  • 195,160
  • 17
  • 277
  • 295
  • Hi, I just updated my question. with the output I got when I tried to write my 'comments' object as csv. That's what is throwing me off. – Shreya Agarwal Jan 16 '20 at 17:03
  • @ShreyaAgarwal That error isn't because it's not a data.frame, that error is because the data.frame contains data that is not atomic and cannot be written to a simple CSV file. You should share a `dput()` of your data so we can see what's inside that object. You should also describe exactly what you want the CSV file to look like given the data. But this has nothing do with the class or type of the object at this point. – MrFlick Jan 16 '20 at 17:05
  • Thanks, I have added the dput() and the desired output as csv. – Shreya Agarwal Jan 16 '20 at 17:12
  • @ShreyaAgarwal The `dput` seems incomplete; it ends with a comma. It only seems to contain atomic vectors so far. – MrFlick Jan 16 '20 at 17:15
  • It's massive, so I pasted just the first half of it, Is there any other way of sharing it? – Shreya Agarwal Jan 16 '20 at 17:17
  • The solution to that is usually to make a smaller, reproducible example. But since you are using the `rtweet` package, that package contains a function called [write_as_csv](https://www.rdocumentation.org/packages/rtweet/versions/0.7.0/topics/write_as_csv) which probably does what you want. Use that rather than `write.csv` – MrFlick Jan 16 '20 at 17:19
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/206084/discussion-between-shreya-agarwal-and-mrflick). – Shreya Agarwal Jan 16 '20 at 17:21
  • So, write_as_csv doesn't support append. And I'm running this code in a for loop, so there will be more tweets from different handles, that id like to store in this csv. Is there any other. way? – Shreya Agarwal Jan 16 '20 at 17:26
  • We seem to be way far away from what your original question was. I suggest you open a new question specifically about `rtweet::write_as_csv` and appending. – MrFlick Jan 16 '20 at 17:28
  • Sure, hanks for your help. I have opened another question on it. – Shreya Agarwal Jan 16 '20 at 17:39