6

I'm a Ruby dev moving over to Clojure and I'm having trouble understanding how to translate the following Java call into Clojure based on the conventions used in the Clojure library Amazonica.

AmazonS3 client = new AmazonS3Client(credentials);
client.setS3ClientOptions(new S3ClientOptions().withPathStyleAccess(true));

The code I have at the moment is:

(ns spurious-aws-sdk-helper.core
  (:use [amazonica.aws.s3]])
  (:require [amazonica.core :refer [ex->map]]))

(def credentials {:access-key "development_access"
                  :secret-key "development_secret"
                  :endpoint "s3.spurious.localhost:49154"
                  :client-config {:protocol "http"}})

(try
  (amazonica.aws.s3/set-s3client-options {:path-style-access true})
  (create-bucket credentials "testing")
  (catch Exception e
    (clojure.pprint/write (ex->map e))))

But I'm getting the following error:

com.amazonaws.http.AmazonHttpClient executeHelper
INFO: Unable to execute HTTP request: testing.s3.spurious.localhost
java.net.UnknownHostException: testing.s3.spurious.localhost

This doesn't look to be correct as it's prefixing the bucket name (testing) onto the hostname. Where as I need the SDK to talk to our local (fake) S3 service (s3.spurious.localhost:49154) using path style.

e.g. like http://s3.spurious.localhost:49154/testing

I think it's because I'm not translating the Java code correctly...

(amazonica.aws.s3/set-s3client-options {:path-style-access true})

...this is passing a map to set-s3client-options rather than what it should be, which is passing the result of calling withPathStyleAccess(true) on a new instance of S3ClientOptions. But I don't know how to do that here?

Any help would be greatly appreciated.

UPDATE

Here is the latest version of the code (which still doesn't work)...

(ns spurious-aws-sdk-helper.core
  (:use [amazonica.aws.s3])
  (:require [amazonica.core :refer [ex->map]]))

(def credentials {:access-key "development_access"
                  :secret-key "development_secret"
                  :endpoint "s3.spurious.localhost:49154"
                  :client-config {:protocol "http"}})

(try
  (amazonica.aws.s3/set-s3client-options 
    (. (com.amazonaws.services.s3.S3ClientOptions.) setPathStyleAccess true))
  (create-bucket credentials "testing")
  (catch Exception e
    (clojure.pprint/write (ex->map e))))
Stéphane Bruckert
  • 21,706
  • 14
  • 92
  • 130
Integralist
  • 5,899
  • 5
  • 25
  • 42

2 Answers2

2

Your first call to set up the S3ClientOptions is the correct one.

I think you're confusing what that setting does. It changes the pre-signed URL generation from using the Virtual Hosted Style https://my_bucket_name.s3.amazonaws.com/my_key to the Path Style https://s3.amazonaws.com/my_bucket_name/my_key

This is necessary if your bucket name contains dots, as the virtual hosted style breaks Amazon's SSL certificate identification (i.e. it only supports the wildcard *.s3.amazonaws.com)

So if you're generating pre-signed URLs on a bucket name containing dots in Clojure e.g.

(s3/generate-presigned-url bucket key (-> 6 hours from-now)))

You will first need to set the path style using

(amazonica.aws.s3/set-s3client-options {:path-style-access true})
stukennedy
  • 1,088
  • 1
  • 9
  • 25
  • thanks for the feedback, I originally asked this question 7+ months ago and so I managed to get it figured out in the end (see specifically https://github.com/Integralist/spurious-clojure-aws-sdk-helper/blob/master/src/spurious_aws_sdk_helper/s3.clj#L14). If anyone is interested the full working project is here: https://github.com/Integralist/spurious-clojure-aws-sdk-helper – Integralist Sep 28 '15 at 14:54
  • why did you not accept my answer and upvote it? If I answered the question correctly first, it's hardly fair to then go in later, post an almost identical answer, accept that and not even give mine an upvote. This should've been the accepted answer; not the one you posted later. I've done the same thing previously, where I figured the problem independently before someone responded. But I still gave them the accepted answer on SO. – stukennedy Feb 03 '16 at 11:57
  • I didn't believe your answer to be the correct one as I was already setting the path-style in the correct way (as you can see in the details of my original question). The actual error turned out to be elsewhere in my code which if you look at my response to you September last year I mention that I figured out the issue myself eventually, but I wanted to thank you for your feedback nonetheless. – Integralist Feb 05 '16 at 11:05
2

Thanks to @stukennedy for responding to the open question. I should have come back long ago and updated this space with the actual solution which I managed to figure out and implement ~8months ago (Feb 2nd 2015):

(ns spurious-aws-sdk-helper.s3
  (:use [amazonica.aws.s3])
  (:require [spurious-aws-sdk-helper.utils :refer [endpoint cred]]))

(defn resource [type]
  (endpoint type :spurious-s3))

(defn setup
  ([type]
   (set-s3client-options (cred (resource type)) :path-style-access true))
  ([type name]
   (try
     (let [credentials (cred (resource type))]
       (set-s3client-options credentials :path-style-access true)
       (create-bucket credentials name))
     (catch Exception e
       (prn "S3 Error: chances are you're creating a bucket that already exists")))))

You can find the full details here: https://github.com/Integralist/spurious-clojure-aws-sdk-helper if you need more context

If I remember correctly, I needed to pass credentials through to set-s3client-options (without them it wouldn't work)

Integralist
  • 5,899
  • 5
  • 25
  • 42