Questions tagged [alpakka]

Alpakka is the collective name for various Akka Streams connectors, integration patterns, and data transformations.

Alpakka is a community-driven initiative that provides connectors, integration patterns, and data transformations that are built with Akka Streams. This toolkit is meant to be a "modern alternative to Apache Camel" (hence its name, which is a homophone of "alpaca," a relative of the camel, and was first used as a code name for an old akka-camel module).

From an introductory blog post:

Akka Streams already has a lot that are useful for integrations. Defining processing pipelines is what the Akka Streams DSL is all about and that is exactly what you need for operating on streaming data that cannot fit in memory as a whole. It handles backpressure in an efficient non-blocking way that prevents out-of-memory errors, which is a typical problem when using unbounded buffering with producers that are faster than consumers.

Connectors:

  • AMQP
  • Apache Geode
  • AWS DynamoDB
  • AWS Kinesis
  • AWS Lambda
  • AWS S3
  • AWS SNS
  • AWS SQS
  • Azure Storage Queue
  • Cassandra
  • Elasticsearch
  • File
  • FTP
  • Google Cloud Pub/Sub
  • HBase
  • IronMq
  • JMS
  • MongoDB
  • MQTT
  • Server-sent Events (SSE)
  • Slick (JDBC)
  • Spring Web

Integration Patterns:

  • Splitter

Data Transformations:

  • Parsing Lines
  • JSON
  • Compressing/decompressing
  • Comma-Separated Values (CSV)
  • RecordIO Framing
  • Extensible Markup Language (XML)

Additional Links:

217 questions
0
votes
0 answers

Elastic search remove multiple types in one index

One of our application was designed such a way that we have 7 indices each per day (7 days worth), and each indices has mapping has mutliple type say userId, but the fields are all the same per type. It was designed long back (ES2.x) version.…
nocturnal
  • 395
  • 2
  • 6
  • 15
0
votes
2 answers

How does slick profile work with respect to slick DB

I am having trouble to understand how to use slick profile. My problem: I am trying to use Slick with Akka-stream via the Alpakka JDBC plugin. The example given online is as follows: #Load using SlickSession.forConfig("slick-h2") slick-h2 { …
MaatDeamon
  • 9,532
  • 9
  • 60
  • 127
0
votes
1 answer

Alpakka Kafka Unable to define Batch

// #atLeastOnceBatch Consumer.Control control = Consumer.committableSource(consumerSettings, Subscriptions.topics("topic1")) .mapAsync(1, msg -> business(msg.record().key(),…
0
votes
1 answer

S3 connector Issue with Akka stream - Alpakka

We are using Alpakka s3 connector to connect to s3 bucket from local system within VPC and getting error as below ,if we use our tradition aws client library we are able to connect to s3 and download file , I am also attaching sample code we are…
Learner
  • 45
  • 1
  • 6
0
votes
1 answer

Alpakka S3Client throws OptionVal$.contains NoSuchMethodError

I am trying to use the Alpakka S3 connector When I instantiate the S3Client I get the exception Exception in thread "main" java.lang.NoSuchMethodError: akka.util.OptionVal$.contains$extension(Ljava/lang/Object;Ljava/lang/Object;) It occurs at …
Daniel
  • 1,522
  • 1
  • 12
  • 25
0
votes
1 answer

Google Pub/Sub Subscriber does not receive messages

First off, I'm not experienced with Akka so I'm really bad in debugging this on my own. I tried the example from here, and publishing messages works (which means credentials work), but no messages are emitted. The service account is granted all…
Daniel
  • 874
  • 10
  • 31
0
votes
1 answer

How to define AmqpSource to subscribe to multiple exchange?

Right now i am subscribing to single exchange using AmqpSource.atMostOnceSource( NamedQueueSourceSettings(..)) I want to be able to subscribe to multiple exchange. Can anyone help me with this?
nocturnal
  • 395
  • 2
  • 6
  • 15
0
votes
1 answer

Akka: How to extract a value in one graph stage and use it in the next

I am using Alpakka and Akka to process a CSV file. Since I have a bunch of CSV files that have to be added to the same stream, I would like to add a field that contains information from the file name or request. Currently I have something like this:…
Falk Schuetzenmeister
  • 1,497
  • 1
  • 16
  • 35
0
votes
0 answers

Why does Alpakka not use the message pattern?

If Alpakka is intended to be an alternative to Apache Camel then why does it not send some form of standardised Message object with a payload, headers, properties, MIME type etc. like all other integration projects do? (e.g. Mule, Spring…
David Plumpton
  • 1,929
  • 23
  • 31
0
votes
1 answer

Alpakka MongoDB - override MongoSource implementation

I have similar problem as in Alpakka MongoDB - specify type in MongoSource So I implemented my own MongoSource as was in the solution and used it: object MyMongoSource { def apply[T](query: Observable[T]): Source[T, NotUsed] = …
Funny
  • 23
  • 1
  • 4
0
votes
1 answer

Alpakka MongoDB - specify type in MongoSource

I'm currently playing around with Akka Streams and the Alpakka MongoDB connector. Is it possible to specify the type for MongoSource? val codecRegistry = fromRegistries(fromProviders(classOf[TodoMongo]), DEFAULT_CODEC_REGISTRY) private val…
user5853948
0
votes
0 answers

How could you handle failures dowstream in an akka stream?

I'm running an akka stream reading from Kafka and I want to commit messages back to Kafka when the file serialization succeeds. But I don't know how to notify upstream stages about a failure downstream. Right now, I've created a GraphStage with…
tonicebrian
  • 4,715
  • 5
  • 41
  • 65
0
votes
1 answer

Alpakka - Callback when message is processed

How can I make a callback after each message is successfully sent to JMS or fails? val jmsSink = JmsSink.textSink( JmsSinkSettings(connectionFactory).withQueue("My_Queue") ) Source(Stream.from(1)) .map(_.toString) .runWith(jmsSink) More…
Feyyaz
  • 3,147
  • 4
  • 35
  • 50
0
votes
0 answers

Alpakka JMS and Slick in Single Transaction

I'm using Alpakka JMS connector to dequeue items from Oracle AQ, and Slick for database operations, the queue and the tables are on the same schema. I need to make the dequeue and DB operations transactionally, so if one fails, the other will also…
Feyyaz
  • 3,147
  • 4
  • 35
  • 50
0
votes
1 answer

Alpakka JMS Transaction

I'm playing with Alpakka and its JMS connector to dequeue data from Oracle AQ. I could come up with the very basic implementation below by following this guide. My question is how I can make it transactional, so that I can guarantee that my message…
Feyyaz
  • 3,147
  • 4
  • 35
  • 50
1 2 3
14
15