aiobotocore s3 file upload it seems things not getting through

Question

I am uploading an image and its thumb to an s3-bucket using simple aiobotocore usecase. Traceback is below. It seems things are not getting through. Could someone help sorting out whats went wrong here. Iam using aiohttp with this usecase.

session = aiobotocore.get_session()
async with session.create_client('s3', region_name='ap-south-1',
                        aws_secret_access_key=key,
                        aws_access_key_id=id) as client:
                                         
    img_resp = await client.put_object(Bucket=bucket,
                                        Key=key_img,
                                        Body=img)
                                
     thumb_resp = await client.put_object(Bucket=bucket,
                                        Key=key_thumb,
                                        Body=thumb)
return {img_resp, thumb_resp}

Traceback: Seems nothing convincing to me

2020-07-28 13:19:03,834 DEBUG botocore.hooks Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2020-07-28 13:19:03,838 DEBUG botocore.hooks Changing event name from before-call.apigateway to before-call.api-gateway
2020-07-28 13:19:03,839 DEBUG botocore.hooks Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2020-07-28 13:19:03,840 DEBUG botocore.hooks Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter--domain.Search.complete-section
2020-07-28 13:19:03,850 DEBUG aiobotocore.credentials Looking for credentials via: env, .....
2020-07-28 13:19:03,860 DEBUG botocore.utils Metadata service returned non-200 response with status code of 404 for url: http://169.254.169.254/latest/meta-data/iam/security-credentials/, content body: <?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head>
  <title>404 - Not Found</title>
 </head>
 <body>
  <h1>404 - Not Found</h1>
 </body>
</html>

2020-07-28 13:19:03,861 DEBUG aiobotocore.utils Max number of attempts exceeded (1) when attempting to retrieve data from metadata service.
2020-07-28 13:19:03,862 DEBUG botocore.loaders Loading JSON file: /home/ubuntu/app/aioenv/lib/python3.6/site-packages/botocore/data/endpoints.json
2020-07-28 13:19:03,872 DEBUG botocore.hooks Event choose-service-name: calling handler <function handle_service_name_alias at 0x7fbd66934950>
2020-07-28 13:19:03,879 DEBUG botocore.loaders Loading JSON file: /home/ubuntu/app/aioenv/lib/python3.6/site-packages/botocore/data/s3/2006-03-01/service-2.json
2020-07-28 13:19:03,893 DEBUG botocore.endpoint Setting s3 timeout as (60, 60)
2020-07-28 13:19:03,894 DEBUG botocore.loaders Loading JSON file: /home/ubuntu/app/aioenv/lib/python3.6/site-packages/botocore/data/_retry.json
2020-07-28 13:19:03,894 DEBUG botocore.client Registering retry handlers for service: s3
2020-07-28 13:19:03,896 DEBUG botocore.hooks Event before-parameter-build.s3.PutObject: calling handler <function validate_ascii_metadata at 0x7fbd668d2c80>
2020-07-28 13:19:03,896 DEBUG botocore.hooks Event before-parameter-build.s3.PutObject: calling handler <function sse_md5 at 0x7fbd668d2158>
2020-07-28 13:19:03,897 DEBUG botocore.hooks Event before-parameter-build.s3.PutObject: calling handler <function convert_body_to_file_like_object at 0x7fbd668d4598>
2020-07-28 13:19:03,897 DEBUG botocore.hooks Event before-parameter-build.s3.PutObject: calling handler <function validate_bucket_name at 0x7fbd668d20d0>
2020-07-28 13:19:03,897 DEBUG botocore.hooks Event before-parameter-build.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7fbd54409a20>>
2020-07-28 13:19:03,897 DEBUG botocore.hooks Event before-parameter-build.s3.PutObject: calling handler <bound method S3ArnParamHandler.handle_arn of <botocore.utils.S3ArnParamHandler object at 0x7fbd543c3080>>
2020-07-28 13:19:03,898 DEBUG botocore.hooks Event before-parameter-build.s3.PutObject: calling handler <function generate_idempotent_uuid at 0x7fbd66943c80>
2020-07-28 13:19:03,898 DEBUG botocore.hooks Event before-call.s3.PutObject: calling handler <function conditionally_calculate_md5 at 0x7fbd668d2048>
2020-07-28 13:19:03,899 DEBUG botocore.hooks Event before-call.s3.PutObject: calling handler <function add_expect_header at 0x7fbd668d2400>
2020-07-28 13:19:03,899 DEBUG botocore.handlers Adding expect 100 continue header to request.
2020-07-28 13:19:03,899 DEBUG botocore.hooks Event before-call.s3.PutObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7fbd54409a20>>
2020-07-28 13:19:03,900 DEBUG botocore.hooks Event before-call.s3.PutObject: calling handler <function inject_api_version_header_if_needed at 0x7fbd668d46a8>
2020-07-28 13:19:03,900 DEBUG botocore.endpoint Making request for OperationModel(name=PutObject) with params: {'url_path': '/c-pix/imgs/d6ef8.jpg', 'query_string': {}, 'method': 'PUT', 'headers': {'x-amz-meta-mappuser': '63bd1', 'User-Agent': 'Botocore/1.15.32 Python/3.6.9 Linux/5.3.0-1030-aws', 'Content-MD5': '8fgw==', 'Expect': '100-continue'}, 'body': <_io.BytesIO object at 0x7fbd543c2258>, 'url': 'https://s3.cc.com/cpix/imgs/c6ef8.jpg', 'context': {'client_region': 'ap-south-1', 'client_config': <aiobotocore.config.AioConfig object at 0x7fbd544095c0>, 'has_streaming_input': True, 'auth_type': None, 'signing': {'bucket': 'cr-pix'}}}
2020-07-28 13:19:03,901 DEBUG botocore.hooks Event request-created.s3.PutObject: calling handler <bound method AioRequestSigner.handler of <aiobotocore.signers.AioRequestSigner object at 0x7fbd54409588>>
2020-07-28 13:19:03,901 DEBUG botocore.hooks Event choose-signer.s3.PutObject: calling handler <function set_operation_specific_signer at 0x7fbd66943b70>
2020-07-28 13:19:03,901 DEBUG botocore.hooks Event before-sign.s3.PutObject: calling handler <bound method S3EndpointSetter.set_endpoint of <botocore.utils.S3EndpointSetter object at 0x7fbd543c30f0>>
2020-07-28 13:19:03,902 DEBUG botocore.utils Defaulting to S3 virtual host style addressing with path style addressing fallback.
2020-07-28 13:19:03,902 DEBUG botocore.utils Checking for DNS compatible bucket for: https://s3.ap-south-1.amazonaws.com/c-pix/imgs/c0d6ef8.jpg
2020-07-28 13:19:03,902 DEBUG botocore.utils URI updated to: https://c-pix.s3.ap-south-1.amazonaws.com/imgs/c0d6ef8.jpg

Thanks in advance

From the error trace, it looks the program isn't using the AWS keys you have set and instead it's querying for the Metadata instance service on the EC2 machine `http://169.254.169.254/latest/meta-data/iam/security-credentials/` Metadata service returned non-200 response. You should check your validity of keys once. — Shivankar, Jul 28 '20 at 15:26
@Shivankar this key was working perfectly with development server. — binrebin, Jul 28 '20 at 16:00
ok you could try configuring the keys by running `aws configure` and then try executing the program? What exactly is the error, are the files not being uploaded? — Shivankar, Jul 28 '20 at 16:33
@Shivankar files are not uploaded and my upstream server debug trace says its stuck there and not evicting any response. — binrebin, Jul 28 '20 at 17:40

aiobotocore s3 file upload it seems things not getting through

0 Answers0