I text mined twitter data with the rtweet
package, but after saving it to a data frame I am not able to use packages such as ggplot2
to their full potential. What am I doing wrong?
I text mined some twitter data using the rtweet
library, and did so in parts so I do not exceed by API limit. After collecting all the data I needed I combined it all into one data frame. I downloaded the dplyr
and ggplot2
packages and wanted to visualise tweets over time, but time time variable is not recognised when it comes from the data frame. But if I use one of the batches of mined data with its original name, it is recognised as a time variable and plotted nicely. This is the code I used to mine the data, save it to data frames and combine them all into one data frame.
##data splits used for mining - 7/8 companies at a time
#1st batch
food_comp1 <- get_timelines(c("accentcatering","angussoftfruits","wmbarrowcliffes","brewdog","brewhouse","capestonefarm","stpierregroupe"), n=3200)
#2nd batch
food_comp2 <- get_timelines(c("cavedirect","brockmoor","cherryfieldltd","fazendagroup","ddcfoods","dairypartners","drakeandmorgan"), n=3200)
#3rd batch
food_comp3 <- get_timelines(c("drinkwarehouse","etsteas","fentimansltd"), n=3200)
#4th batch
food_comp4 <- get_timelines(c("goustocooking","edinburgh_gin","innisandgunn","grapetreefoods","kkfinefoods","finnebrogue","thewhiskyshop"), n=3200)
#5th batch
food_comp5 <- get_timelines(c("onarollsandwich","potsandco","primacheese","purecircle","silburyoils","thealchemistuk"), n=3200)
#6th batch
food_comp6 <- get_timelines(c("artisan_glasgow","thebigprawnco","foodfellas","freshfoodco","wagyurestaurant","charlesfaram","wenzelsthebaker","westerrosssalmo"), n=3200)
#7th batch
food_comp7 <- get_timelines(c("elitefoods_","fevertreemixers","gordon_macphail","gosh_freefrom","specialitydrink"), n=3200)
#merging the datasets into 1
comp <- rbind(food_comp1,food_comp2,food_comp3,food_comp4,food_comp5,food_comp6,food_comp7)
write_as_csv(comp, file_name = "comp", prepend_ids = TRUE, na="", fileEncoding = "UTF-8")
comp <- read.csv("comp.csv", header = TRUE)
View(comp)
#creating a subset with the variables I need
compsubset <- subset(comp, select=c("created_at","screen_name","text","display_text_width","favorite_count","retweet_count","hashtags","media_type","lang"))
write_as_csv(compsubset, file_name = "compsubset", prepend_ids = TRUE, na="", fileEncoding = "UTF-8")
##final dataset with 9 variables
compsubset <- read.csv("compsubset.csv", header=TRUE)
#attemting to create a ggplot with created_at as the time variable (displays year-date-time)
ggplot(data = compsubset, aes(x = created_at)) +
geom_histogram(aes(fill = ..count..)) +
theme(legend.position = "none") +
xlab("Time") + ylab("Number of tweets") +
scale_fill_gradient(low = "midnightblue", high = "aquamarine4")
Error: StatBin requires a continuous x variable: the x variable is discrete. Perhaps you want stat="count"?
The same code with one of the mined batches before converted to a data frame
ggplot(data = food_comp1, aes(x = created_at)) +
geom_histogram(aes(fill = ..count..)) +
theme(legend.position = "none") +
xlab("Time") + ylab("Number of tweets") +
scale_fill_gradient(low = "midnightblue", high = "aquamarine4")
After this last ggplot I got a nice bar chart with tweets over time
Similar thing happens when I try to use dplyr
to filter tweets by time, but works fine with the mined data before putting into dataframe
comptrial %>%
dplyr::filter(created_at >= "2018-01-01") %>%
dplyr::filter(created_at < "2019-01-01") %>%
dplyr::group_by(screen_name) %>%
ts_plot("days", trim = 1L) +
ggplot2::geom_point() +
ggplot2::theme_minimal() +
ggplot2::theme(
legend.title = ggplot2::element_blank(),
legend.position = "bottom",
plot.title = ggplot2::element_text(face = "bold")) +
ggplot2::labs(
x = NULL, y = NULL,
title = "Frequency of Twitter statuses posted by SMEs",
subtitle = "Twitter status (tweet) counts
caption = "\nSource: Data collected from SME's REST API via rtweet"
)
Error in seq.POSIXt(data[[dtvar]][1], data[[dtvar]][length(data[[dtvar]])], : 'to' must be of length 1 In addition: Warning messages: 1: In Ops.factor(created_at, "2018-01-01") : ‘>=’ not meaningful for factors 2: Factor
screen_name
contains implicit NA, consider usingforcats::fct_explicit_na
3: Factorscreen_name
contains implicit NA, consider usingforcats::fct_explicit_na
4: Factorscreen_name
contains implicit NA, consider usingforcats::fct_explicit_na