Stack Overflow
Stack Overflow

Questions tagged [apache-apex]

Apex is a YARN-native platform that unifies stream and batch processing. It processes big data in-motion in a way that is highly scalable, highly performant, fault tolerant, stateful, secure, distributed, and easily operable.

Apache Apex is an open source platform for a unified stream and batch processing. It runs natively on YARN. Apache Apex itself is divided into two core modules

Developing application on Apex

To start with, documentation is available at: http://apex.apache.org/docs.html

51 questions
2
votes
3 answers

Can an Input Operator be used in the middle of a DAG in Apache Apex

All examples of Apex say that the first operator of the DAG should be an input operator. Can this operator appear in the middle of the DAG somewhere. Consider a case in which I have data to be fetched from the database, based on some data that has…
frewper
  • 1,385
  • 6
  • 18
  • 44
2
votes
3 answers

How to use Apache Apex Malhar RabbitMQ operator in DAG

I have an Apache Apex application DAG which reads RabbitMQ message from a queue. Which Apache Apex Malhar operator should I use? There are several operators but it's not clear which one to use and how to use it.
Ab Gupta
  • 21
  • 1
2
votes
2 answers

How to make batch processing with Apex?

How can I create batch processing application with Apache Apex? All the examples I've found were streaming applications, which means they are not ending and I would like my app to close once it has processed all the data. Thanks
Krever
  • 1,371
  • 1
  • 13
  • 32
2
votes
3 answers

Use Apex without DataTorrent RTS

Is it possible to use Apache Apex without DataTorrent RTS? It seems possible, but I couldn't find any tutorial about installing own build of apex sources(to run, you need at leas apex binary). I see that Apex CLI is available after build in…
Krever
  • 1,371
  • 1
  • 13
  • 32
2
votes
2 answers

How to pass arguments to application.java class in Apache Apex?

How can I pass arguments to Application.java class ? Like how we pass (String[] args) in main method in java ? Couldn’t find any references. Can someone please share knowledge on this ? Thanks a lot.
Raja
  • 513
  • 5
  • 18
2
votes
1 answer

How to restart Apache Apex application?

From the apex documentation, it is clear that an app launched with apache apex can be killed or shutdwon using the commands: kill-app & shutdown-app respectively. But, when the application is turned off (shutdown/kill), how to restart it from its…
PradeepKumbhar
  • 3,361
  • 1
  • 18
  • 31
2
votes
1 answer

Difference between Shutdown-app & kill-app in Apache Apex CLI commands

can someone help me understand, what is the difference between Shutdown-app & kill-app in Apache APEX CLI commands ?
Raja
  • 513
  • 5
  • 18
2
votes
1 answer

Controlling YARN memory settings within Apache Apex memory model

I am seeing the following log from the web UI occasionally when my operators are getting killed. Is there any way I can control the memory settings that are used to communicate with YARN when negotiating a container ? How does the typical YARN…
user6147934
2
votes
2 answers

What is the operator lifecycle in Apache Apex?

Repost from users@apex.incubator.apache.org When is the constructor of an operator called ? The docs say once in the lifetime of an operator but I was wondering whether the definition of "lifetime" spans across start/stop/crash ( because of a coding…
user6147934
2
votes
2 answers

Does Apache Apex rely on HDFS or does it have its own file system?

I understand that Apache Apex runs on Hadoop and YARN. Does it utilize HDFS for persistence and replication to protect against data loss? or does it have its own?
user6147934
2
votes
1 answer

Out of order processing in Apache Apex

The apache-apex documentation states that the order of events in processors is preserved (based on how they are emitted), but does that mean that there is no event-time concept that allows out-of-order processing of events? Moreover, can events be…
brusli
  • 79
  • 9
1
vote
1 answer

Window ID resets to 0 after it reaches 59 in DTconsole

I am currently working on apache apex. My application gets reset after window ID 59 is reached and starts again from 0. I get an error which says that there is only 1 datanode running and no nodes are excluded in this operation
neeraj goidani
  • 13
  • 4
1
vote
1 answer

Logging in Apache Apex using log4j in docker RTS sandbox

I would like to understand how I can see my application logs using log4j in Apache Apex when using the official RTS Test Sandbox in docker. To be more specific, I would like to know where Apex stores application-specific log files inside of the…
ice_chrysler
  • 2,633
  • 1
  • 21
  • 27
1
vote
3 answers

Need architecture hint: Data replication into the cloud + data cleansing

I need to sync customer data from several on-premise databases into the cloud. In a second step, the customer data there needs some cleanup in order to remove duplicates (of different types). Based on that cleansed data I need to do some data…
Andreas Hof
  • 151
  • 1
  • 6
1
vote
1 answer

Maven archetype error while creating a new Apache Apex project

I am following the documentation related to the Apache Apex which helps create a startup project. https://apex.apache.org/docs/apex-3.4/apex_development_setup/#creating-new-apex-project While executing the maven command, @echo off @rem Script for…
frewper
  • 1,385
  • 6
  • 18
  • 44
1
2
3 4