When you read in the data with read.table (or read.csv? - you didn't specify), add the argument stringsAsFactors = FALSE. Then you will get character data instead.
If you are expecting integers for the column then you must have data that is not interpretable as integers, so convert to numeric after you've read it.
txt <- c("x,y,z", "1,2,3", "a,b,c")
d <- read.csv(textConnection(txt))
sapply(d, class)
x y z
##"factor" "factor" "factor"
## we don't want factors, but characters
d <- read.csv(textConnection(txt), stringsAsFactors = FALSE)
sapply(d, class)
# x y z
#"character" "character" "character"
## convert x to numeric, and wear NAs for non numeric data
as.numeric(d$x)
#[1] 1 NA
#Warning message:
#NAs introduced by coercion
Finally, if you want to ignore these input details and extract the integer levels from the factor use e.g. as.numeric(levels(d$x))[d$x], as per "Warning" in ?factor.