I'm trying to use Alpakka S3 to connect to a minio instance in order to store files, but I'm running into problems since I upgraded the library version from 1.1.2 to 2.0.0.
Here is a simple service class with just two methods that try to create a bucket. I tried both approaches, first loading alpakka settings from local config file (application.conf in my case) and second via direct creation of the settings via S3Ext.
Both methods fail and I'm not sure about the problem. Regarding the error, it seems the settings are not loaded correctly, but I don't know what I'm doing wrong here.
What I'm using:
- play framework 2.8.1
- scala 2.13.2
- akka-stream-alpakka-s3 2.0.0
Here is the service class:
package services
import akka.actor.ActorSystem
import akka.stream.alpakka.s3._
import akka.stream.alpakka.s3.scaladsl.S3
import akka.stream.scaladsl.Sink
import akka.stream.{Attributes, Materializer}
import javax.inject.{Inject, Singleton}
import software.amazon.awssdk.auth.credentials.{AwsBasicCredentials, AwsCredentials, AwsCredentialsProvider}
import software.amazon.awssdk.regions.Region
import software.amazon.awssdk.regions.providers.AwsRegionProvider
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
@Singleton
class AlpakkaS3PlaygroundService @Inject()(
materializer: Materializer,
system: ActorSystem,
) {
def makeBucket(bucketName: String): Future[String] = {
S3.makeBucket(bucketName)(materializer) map { _ =>
"bucket created"
}
}
def makeBucket2(bucketName: String): Future[String] = {
val s3Host = "http://localhost:9000"
val s3AccessKey = "access_key"
val s3SecretKey = "secret_key"
val s3Region = "eu-central-1"
val credentialsProvider = new AwsCredentialsProvider {
override def resolveCredentials(): AwsCredentials = AwsBasicCredentials.create(s3AccessKey, s3SecretKey)
}
val regionProvider = new AwsRegionProvider {
override def getRegion: Region = Region.of(s3Region)
}
val settings: S3Settings = S3Ext(system).settings
.withEndpointUrl(s3Host)
.withBufferType(MemoryBufferType)
.withCredentialsProvider(credentialsProvider)
.withListBucketApiVersion(ApiVersion.ListBucketVersion2)
.withS3RegionProvider(regionProvider)
val attributes: Attributes = S3Attributes.settings(settings)
S3.makeBucketSource(bucketName)
.withAttributes(attributes)
.runWith(Sink.head)(materializer) map { _ =>
"bucket created"
}
}
}
The config in application.conf looks like this:
akka.stream.alpakka.s3 {
aws {
credentials {
provider = static
access-key-id = "access_key"
secret-access-key = "secret_key"
}
region {
provider = static
default-region = "eu-central-1"
}
}
endpoint-url = "http://localhost:9000"
}
If the first method of the service is used (makeBucket(...)) I see this error:
SdkClientException: Unable to load region from any of the providers in the chain software.amazon.awssdk.regions.providers.DefaultAwsRegionProviderChain@34cb16dc:
[software.amazon.awssdk.regions.providers.SystemSettingsRegionProvider@804e08b: Unable to load region from system settings. Region must be specified either via environment variable (AWS_REGION) or system property (aws.region)., software.amazon.awssdk.regions.providers.AwsProfileRegionProvider@4d5f4b4d: No region provided in profile: default, software.amazon.awssdk.regions.providers.InstanceProfileRegionProvider@557feb58: Unable to contact EC2 metadata service.]
The error message is quite precise and I know what's wrong, but I just don't know what to do because I specified the settings as outlined in the documentation. Any idea?
In the second method of the service (makeBucket2(...)) I try to explicitly set the S3 setting, but this does not seem to work either. The error looks like this:
play.api.http.HttpErrorHandlerExceptions$$anon$1: Execution exception[[S3Exception: 404 page not found
]]
at play.api.http.HttpErrorHandlerExceptions$.throwableToUsefulException(HttpErrorHandler.scala:335)
at play.api.http.DefaultHttpErrorHandler.onServerError(HttpErrorHandler.scala:253)
at play.core.server.AkkaHttpServer$$anonfun$2.applyOrElse(AkkaHttpServer.scala:424)
at play.core.server.AkkaHttpServer$$anonfun$2.applyOrElse(AkkaHttpServer.scala:420)
at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:453)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:92)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94)
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:92)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:47)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:47)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
Caused by: akka.stream.alpakka.s3.S3Exception: 404 page not found
Here it looks like the defined setting are not taken into account at all, since the service does not seem to be found. This is actually the approach I used in former version of my software where I used akka-stream-alpakka-s3 version 1.1.2 and it worked as expected.
Of course I want to use Alpakka S3 not only to create buckets, but for the sake of this showcase and to outline my problem, I kept it simple with only this example. I guess, if this problem is solved, all other methods provided by alpakka will just work.
I really red the documentation several times but I'm still not able to fix this issue, so I hope someone here can help me.
