Questions tagged [alpakka]

Alpakka is the collective name for various Akka Streams connectors, integration patterns, and data transformations.

Alpakka is a community-driven initiative that provides connectors, integration patterns, and data transformations that are built with Akka Streams. This toolkit is meant to be a "modern alternative to Apache Camel" (hence its name, which is a homophone of "alpaca," a relative of the camel, and was first used as a code name for an old akka-camel module).

From an introductory blog post:

Akka Streams already has a lot that are useful for integrations. Defining processing pipelines is what the Akka Streams DSL is all about and that is exactly what you need for operating on streaming data that cannot fit in memory as a whole. It handles backpressure in an efficient non-blocking way that prevents out-of-memory errors, which is a typical problem when using unbounded buffering with producers that are faster than consumers.

Connectors:

  • AMQP
  • Apache Geode
  • AWS DynamoDB
  • AWS Kinesis
  • AWS Lambda
  • AWS S3
  • AWS SNS
  • AWS SQS
  • Azure Storage Queue
  • Cassandra
  • Elasticsearch
  • File
  • FTP
  • Google Cloud Pub/Sub
  • HBase
  • IronMq
  • JMS
  • MongoDB
  • MQTT
  • Server-sent Events (SSE)
  • Slick (JDBC)
  • Spring Web

Integration Patterns:

  • Splitter

Data Transformations:

  • Parsing Lines
  • JSON
  • Compressing/decompressing
  • Comma-Separated Values (CSV)
  • RecordIO Framing
  • Extensible Markup Language (XML)

Additional Links:

217 questions
1
vote
1 answer

Save DTO in Alpakka Cassandra

Hi I create my own Connector of Cassandra using the datastax drivers. But I´m facing some Memory leaks issues, so I start considering another solutions like Alpakka de lightbend which has a Cassandra connector. But after check the poor documentation…
paul
  • 12,873
  • 23
  • 91
  • 153
1
vote
1 answer

Explicit ack to JMS using Alpakka

I am currently using Akka-Camel for integrating my Akka based application with various message queues. Now that I want to upgrade to the latest version of Akka I see that this integration is now deprecated. I have tried using alpakka-jsm…
Evan M.
  • 403
  • 5
  • 13
1
vote
1 answer

alpakka, JMS and correlationId in jms.Message in response

I've used alpakka(Scala) and JMS connector for sending/receiving messages between client and server. All working perfect, but I don't know - how I can set correlationId in jms.Message in response. Any idea, please. Thank you.
0
votes
1 answer

How to set headers in RabbitMq with Scala

Hi I'm trying to set headers in the rabbit messages with the library akka-stream-alpakka-amqp in scala but I can't be able to make it work and reading the repository I can't see any reference or method to add headers in the messages. I was trying to…
0
votes
0 answers

Is Akka, Alpakka Google Common and Google Cloud Email Sender the efficient and correct way?

Hi im learning how to send and email in Akka with scala. I've been trying to use Alpakka Google Common with Gcloud service account credential and Google Api Client to send the email. I have come across several errors with the data types of the…
0
votes
1 answer

Does Alpakka Kafka Consumer.committablePartitionedSource uses separate kafka consumer per partition

I want to understand if Consumer.committablePartitionedSource uses separate kafka consumer per partition(all my partitions say(8) are assigned to single node)
0
votes
1 answer

S3.getObject is truncating file contents in alpakka 4.0.0

I have a block of code like this: def downloadFilesSource(bucketName: String, primaryKey: String)( implicit ec: ExecutionContext ): Source[(Source[ByteString, NotUsed], String), NotUsed] = S3.listBucket(bucketName, prefix =…
user3468054
  • 610
  • 4
  • 11
0
votes
1 answer

Amqprpc alpakka producer not receiving a response back

I´m currently trying to implement alpakka amqp queues, and specifically I´m trying to use rabbitmq as an RPC mechanism. So what I´m trying to do is to send messages from amqprpc flow, and then the consumer will do some calculation and then the…
0
votes
0 answers

Run a Alpakka Kafka Consumer on Demand in Scala

When all the messages have been handled, I need to shut off the Kafka consumer until the next call. I obviously won't be able to use what I wrote. Any assistance would be greatly valued. val consumerDrainingControl = Consumer .mapAsync(1) {…
0
votes
1 answer

Problems with Incompletely Consumed Akka Streams

We have a service that uses Alpakka 3.0.4 (Scala 2.13) to stream multiple large files from S3, zip them, and send the zipped stream out as HTTP responses. The idea is to start sending out even before zipping is complete, to start zipping even before…
silverberry
  • 786
  • 5
  • 20
0
votes
1 answer

Code not working for FileTailSource in akka Stream

I have a log file that is fed data about every visit for some websites ( it is actually just a python programm that simaulates this ) , i want to count the number of visits per websites . So i m trying to use FileTailSource but it doesnt print…
ach2ashes
  • 39
  • 5
0
votes
1 answer

Akka Stream application using more memory than the jvm's heap

Summary: I have a Java application that uses akka streams that's using more memory than I have specified the jvm to use. The below values are what I have set through the JAVA_OPTS. maximum heap size (-Xmx) = 700MB metaspace (-XX) = 250MB stack size…
Rlupo07
  • 51
  • 1
  • 8
0
votes
1 answer

Benefits of Alpakka Kafka Producer over plain Kafka Producer

I have an application that receives an api request and relays its to a Kafka API Producer. Each request calls the producer to send a message to Kafka. The producer exists throughout the application lifetime and is shared for all…
landau
  • 41
  • 1
  • 6
0
votes
0 answers

Alpakka: google-cloud-pub-sub - One subscriber consuming 60% of messages

Summary: I have an application (subscriber) that I'm scaling to 4 instances. All 4 instances are subscribers of one subscription. For some reason 1 of those instances is consuming 60% of messages on the Pub/Sub subscription (300k+ small messages on…
Rlupo07
  • 51
  • 1
  • 8
0
votes
0 answers

How to pull records from bigquery using Akka Streams Alpakka Big query connector

I have a query for big query native table, currently I'm able to run and get the data and write to sync, but I need to run the same query periodically and fetch the results. My implementation looks something like this: val sqlQuery = s"SELECT name,…
Mathivanan
  • 371
  • 2
  • 16