Stack Overflow
Stack Overflow

Questions tagged [boto3]

Boto 3 - The Amazon Web Services (AWS) SDK for Python

About

Boto is the Amazon Web Services (AWS) software development kit (SDK) for Python, which allows Python developers to write software that makes use of services like Amazon S3 and Amazon EC2. Boto provides an easy to use, object-oriented API as well as low-level direct service access.

Links

7627 questions
79
votes
3 answers

getting the current user account-id in boto3

I need to get the account-id of the 'current user' in a boto3 script. Up to now my best solution is to parse the current user arn: >>> import boto3 >>> account_id = boto3.resource('iam').CurrentUser().arn.split(':')[4] but I was wondering if there…
Stefano M
  • 4,267
  • 2
  • 27
  • 45
76
votes
9 answers

How to read a list of parquet files from S3 as a pandas dataframe using pyarrow?

I have a hacky way of achieving this using boto3 (1.4.4), pyarrow (0.4.1) and pandas (0.20.3). First, I can read a single parquet file locally like this: import pyarrow.parquet as pq path =…
Diego Mora Cespedes
  • 3,605
  • 5
  • 26
  • 33
74
votes
1 answer

Update attribute "timestamp" reserved word

I need to update the timestamp attribute in my dynamodb table using boto3 but the attribute name "timestamp" is a reserved word so it's throwing an error on the SET command. table.update_item( Key={ 'id': item_id }, …
Cread Dotson
  • 982
  • 1
  • 7
  • 18
73
votes
10 answers

Example of update_item in dynamodb boto3

Following the documentation, I'm trying to create an update statement that will update or add if not exists only one attribute in a dynamodb table. I'm trying this response = table.update_item( Key={'ReleaseNumber': '1.0.179'}, …
Dmitry R
  • 2,956
  • 4
  • 25
  • 45
69
votes
4 answers

"The provided key element does not match the schema" error when getting an item from DynamoDB

This is the table partition key setting The table content When I tried to get an item from the table, it prints this error botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the GetItem operation: The…
Keoros
  • 1,337
  • 2
  • 13
  • 23
66
votes
10 answers

How to read image file from S3 bucket directly into memory?

I have the following code import matplotlib.pyplot as plt import matplotlib.image as mpimg import numpy as np import boto3 s3 = boto3.resource('s3', region_name='us-east-2') bucket = s3.Bucket('sentinel-s2-l1c') object =…
Dims
  • 47,675
  • 117
  • 331
  • 600
64
votes
6 answers

How can I easily determine if a Boto 3 S3 bucket resource exists?

For example, I have this code: import boto3 s3 = boto3.resource('s3') bucket = s3.Bucket('my-bucket-name') # Does it exist???
Daniel
  • 8,212
  • 2
  • 43
  • 36
63
votes
5 answers

Download file from AWS S3 using Python

I am trying to download a file from Amazon S3 bucket to my local using the below code but I get an error saying "Unable to locate credentials" Given below is the code I have written: from boto3.session import Session import boto3 ACCESS_KEY =…
Taukheer
  • 1,091
  • 2
  • 13
  • 20
63
votes
4 answers

Writing json to file in s3 bucket

This code writes json to a file in s3, what i wanted to achieve is instead of opening data.json file and writing to s3 (sample.json) file, how do i pass the json directly and write to a file in s3 ? import boto3 s3 = boto3.resource('s3',…
Learning
  • 716
  • 1
  • 7
  • 13
62
votes
4 answers

How do we query on a secondary index of dynamodb using boto3?

Is there a way at all to query on the global secondary index of dynamodb using boto3. I dont find any online tutorials or resources.
ZZzzZZzz
  • 1,800
  • 3
  • 29
  • 51
60
votes
10 answers

Download a folder from S3 using Boto3

Using Boto3 Python SDK, I was able to download files using the method bucket.download_file() Is there a way to download an entire folder?
El Fadel Anas
  • 1,581
  • 2
  • 18
  • 25
60
votes
3 answers

how to copy s3 object from one bucket to another using python boto3

I want to copy a file from one s3 bucket to another. I get the following error: s3.meta.client.copy(source,dest) TypeError: copy() takes at least 4 arguments (3 given) I'am unable to find a solution by reading the docs. Here is my code: …
vishal
  • 1,646
  • 5
  • 28
  • 56
60
votes
4 answers

How to convert a boto3 Dynamo DB item to a regular dictionary in Python?

In Python, when an item is retrieved from Dynamo DB using boto3, a schema like the following is obtained. { "ACTIVE": { "BOOL": true }, "CRC": { "N": "-1600155180" }, "ID": { "S": "bewfv43843b" }, "params": { "M": { …
manelmc
  • 937
  • 2
  • 7
  • 17
60
votes
3 answers

What is the Difference between file_upload() and put_object() when uploading files to S3 using boto3

I'm using boto3 and trying to upload files. It will be helpful if anyone will explain exact difference between file_upload() and put_object() s3 bucket methods in boto3 ? Is there any performance difference? Does anyone among these handles…
Tushar Niras
  • 3,654
  • 2
  • 22
  • 24
59
votes
4 answers

How to query cloudwatch logs using boto3 in python

I have a lambda function that writes metrics to Cloudwatch. While, it writes metrics, It generates some logs in a log-group. INFO:: username: simran+test@example.com ClinicID: 7667 nodename: MacBook-Pro-2.local INFO:: username:…
systemdebt
  • 4,589
  • 10
  • 55
  • 116
1 2
3
…
99 100