Questions tagged [smart-open]
8 questions
2
votes
1 answer
ValueError: the bucket does not exist, or is forbidden for access 'An error occurred (AccessDenied) when calling the CreateMultipartUpload
I am using python library smart_open to upload file (it would be big files) from python script to S3 bucket
Bucket has policy enforcing SSE with KMS
{
"Version": "2012-10-17",
"Id": "PutObjPolicy",
"Statement": [
{
…
bensiu
- 24,660
- 56
- 77
- 117
1
vote
1 answer
mock_open with variable read_data
I tried looking through the similar questions, but couldn't find an answer that answers my question.
I have a function like so:
from smart_open import open
import pandas as pd
def foo():
with open('file1.txt', 'r') as f:
df1 = pd.read_csv(f)
…
Lukas Hestermeyer
- 830
- 1
- 7
- 19
0
votes
2 answers
Python smart_open throws NotImplementedError with code from documentation
I am trying to write to a .txt file stored in Azure Blob Storage using the smart_open module. However, when running the example code provided in the documentation, I encounter the following error:
NotImplementedError: http support for mode 'wb' not…
Pablo Castrejón
- 21
- 3
0
votes
0 answers
Using smart-open with AWS lambda S3 access error
I'm trying to stream a file from my requests.get to my s3 using smart-open within my lambda. My lambda role has proper access to the s3, needed action of "s3:GetObject","s3:PutObject", and kms key to decrypted bucket.
I'm getting error message the…
Jack
- 5
- 4
0
votes
1 answer
connecting to AWS S3 using boto3 or smart_open - debugging connection error
My goal is to upload objects to S3, I have been trying with both smart_open and boto3 libraries with no success.
I don't know much about configuring IAM policies or Access points in S3; but finding very hard to debug and understand how to pass…
user305883
- 1,635
- 2
- 24
- 48
0
votes
1 answer
FileResponse with file opened using smart-open
I'm trying to let a user download a file from my webpage. The file is in an S3 bucket I'm accessing using smart-open. The issue I'm having is how to combine that with FileReader. Presently I'm getting a TypeError: "expected str, bytes or os.PathLike…
user17817249
- 83
- 7
-1
votes
1 answer
Use smart_open to download a .gz stream from http and upload to s3 bucket
I would like to stream download a .txt.gz file from http and stream upload to an s3 bucket, I've gotten to this but it doesn't work, what am I missing?
from smart_open import open as sopen
chunk_size = (16 * 1024 * 1024)
http_url =…
ffi23
- 21
- 5
-1
votes
1 answer
use boto3 session when opening s3 url
I see a lot of code that uses an S3 bucket url to open a file. I would like to use smart open to open a compressed file
session = boto3.Session(ID, pass)
file = open("s3://bucket/file.txt.gz", transport_params=dict(session=session),…
A Simple Programmer
- 494
- 4
- 15