I am at a loss as to which polars interface I would pass s3_bytes from json lines file to get a dataframe
impl S3_Operation {
pub async fn new(file: File) -> Self {
let config = aws_config::load_from_env().await;
let client = Client::new(&config);
S3_Operation { client, file }
}
pub async fn download_object(&self) -> Cursor<Bytes> {
let resp = self
.client
.get_object()
.bucket(&self.file.bucket)
.key(&self.file.key)
.send()
.await;
let s3_bytes = resp.unwrap().body.collect().await.unwrap().into_bytes();
I want to get a polars dataframe at the end of day without moving s3_bytes to local storage. What am I missing here. JsonReader has no new fn while LazyJsonLineReader needs a path ...