2

I am trying to UNLOAD a Redshift table to an S3 bucket, but I am getting errors that I can't resolve.

When using 's3://mybucket/' as the destination (which is the documented way to specify the destination), I have an error saying S3ServiceException:The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint..

After some research I have tried to change the destination to include the full bucket url, without success.

All these destinations:

  • 's3://mybucket.s3.amazonaws.com/',
  • 's3://mybucket.s3.amazonaws.com/myprefix',
  • 's3://mybucket.s3.eu-west-2.amazonaws.com/',
  • 's3://mybucket.s3.eu-west-2.amazonaws.com/myprefix'

return this error S3ServiceException:The authorization header is malformed; the region 'eu-west-2' is wrong; expecting 'us-east-1', which is also the error returned when I use a bucket name that doesn't exist.

My Redshift cluster and my s3 buckets all exist in the same region, eu-west-2.

What am I doing wrong?

[appendix]

Full command:

UNLOAD ('select * from mytable')
to 's3://mybucket.s3.amazonaws.com/'
iam_role 'arn:aws:iam::0123456789:role/aws-service-
role/redshift.amazonaws.com/AWSServiceRoleForRedshift'

Full errors:

ERROR:  S3ServiceException:The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.,Status 301,Error PermanentRedirect,Rid 6ADF2C929FD2BE08,ExtRid vjcTnD02Na/rRtLvWsk5r6p0H0xncMJf6KBK
DETAIL:
  -----------------------------------------------
  error:  S3ServiceException:The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.,Status 301,Error PermanentRedirect,Rid 6ADF2C929FD2BE08,ExtRid vjcTnD02Na/rRtLvWsk5r6p0H0xncMJf6KBK
  code:      8001
  context:   Listing bucket=mybucket prefix=
  query:     0
  location:  s3_unloader.cpp:226
  process:   padbmaster [pid=30717]
  -----------------------------------------------

ERROR:  S3ServiceException:The authorization header is malformed; the region 'eu-west-2' is wrong; expecting 'us-east-1',Status 400,Error AuthorizationHeaderMalformed,Rid 559E4184FA02B03F,ExtRid H9oRcFwzStw43ynA+rinTOmynhWfQJlRz0QIcXcm5K7fOmJSRcOcHuVlUlhGebJK5iH2L
DETAIL:
  -----------------------------------------------
  error:  S3ServiceException:The authorization header is malformed; the region 'eu-west-2' is wrong; expecting 'us-east-1',Status 400,Error AuthorizationHeaderMalformed,Rid 559E4184FA02B03F,ExtRid H9oRcFwzStw43ynA+rinTOmynhWfQJlRz0QIcXcm5K7fOmJSRcOcHuVlUlhGebJK5iH2L
  code:      8001
  context:   Listing bucket=mybucket.s3.amazonaws.com prefix=
  query:     0
  location:  s3_unloader.cpp:226
  process:   padbmaster [pid=30717]
  -----------------------------------------------

Bucket zone

Cluster zone

Thierry J.
  • 2,148
  • 16
  • 23
  • Try `to 's3://mybucket/filename.extension'` rather than `to 's3://mybucket.s3.amazonaws.com/'` – Yusuf Hassan Jan 18 '18 at 09:43
  • Thanks for the suggestion, but I am still getting the first error. – Thierry J. Jan 18 '18 at 09:54
  • 1
    The `specified endpoint` error normally means that the bucket is in a different region. Check again that the bucket is in the same region as your Redshift cluster. From the documentation: "The Amazon S3 bucket where Amazon Redshift will write the output files must reside in the same region as your cluster." Please Edit your question to show the proof that they are in the same region. – John Rotenstein Jan 18 '18 at 11:14
  • When using `s3://` syntax, do not include the `s3.amazonaws.com` portion of the URI. Just include the bucketname and path. – John Rotenstein Jan 18 '18 at 11:15
  • Yes, I had read that it usually means a region issue, but I have checked multiple times, and I can confirm that the cluster and the bucket exist in the region `eu-west-2` – Thierry J. Jan 18 '18 at 11:35
  • Can you try a different (new) bucket, as a test? Also, does the `AWSServiceRoleForRedshift` have permissions to store data in the bucket? – John Rotenstein Jan 18 '18 at 20:25
  • I have already tried with a new bucket, same problem. The role does have the right permissions, just in case I recreated a new role with full access to S3 and I still have the same problems... – Thierry J. Jan 19 '18 at 13:25
  • I added screenshots showing the region of both the bucket and the cluster. – Thierry J. Jan 22 '18 at 09:37
  • 1
    Make sure the IAM role has `AmazonS3FullAccess` and `AmazonRedshiftFullAccess` policies and, in Trust Relationships, a trusted entity of `The identity provider(s) redshift.amazonaws.com`. – Joe Harris Jan 23 '18 at 13:54
  • Awesome, thanks! Please add a formal answer so I can validate it.. – Thierry J. Jan 25 '18 at 16:14

0 Answers0