Stack Overflow
Stack Overflow

Questions tagged [apache-twill]

Apache Twill is a library for developing distributed applications with Apache Hadoop® YARN.

Apache Twill is an abstraction over Apache Hadoop® YARN that reduces the complexity of developing distributed applications, allowing developers to focus more on their application logic. Apache Twill allows you to use YARN’s distributed capabilities with a programming model that is similar to running threads.

Apache Twill is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by Incubator.

The current version of Twill is 0.7.0-incubating. More information on Twill is available at the Apache Twill Website.

14 questions
4
votes
1 answer

IllegalAccessError for RequestHedgingRMFailoverProxyProvider while launching Apache Twill Application in hadoop cluster after HDP upgrade

I'm trying to launch Apache Twill application from hadoop cluster, the cluster is recently upgraded from HDP 2.2 to HDP 2.5 but I'm getting llegalAccessError for RequestHedgingRMFailoverProxyProvider class . This class is part of…
insanely_sin
  • 986
  • 1
  • 14
  • 22
3
votes
1 answer

Getting "User [dr.who] is not authorized to view the logs for application " while running a YARN application

I'm running a custom Yarn Application using Apache Twill in HDP 2.5 cluster, but I'm not able to see my own container logs (syslog, stderr and stdout) when I go to my container web page: Also the login changes from my kerberos to "dr.who" when I…
insanely_sin
  • 986
  • 1
  • 14
  • 22
1
vote
0 answers

Send arbitrary local jars to YARN container classpath

I'm using Apache Twill (v 0.10) to build YARN application. I've observed that the jars which are not referenced by my application code are not picked up and sent to containers' classpath. I checked YarnTwillPreparer class to see how the dependencies…
RRM
  • 2,495
  • 29
  • 46
1
vote
0 answers

Deploy multiple instances of war on Hadoop through Apache Twill

I've a web app archive (war) file which is deployed on Tomcat. It contains some REST services. To scale, I want to deploy these REST services (either as war or as jar) on Hadoop cluster through Apache Twill. I want to use Jetty instead of Tomcat. I…
RRM
  • 2,495
  • 29
  • 46
1
vote
1 answer

Supply different JVM options to different runnable in Apache Twill

In a Twill application lets say I have two types of runnables. One runnable has a 3GB container size and a different runnable has a 6GB container size. Now I want to set different -Xms option for these two runnable types. How do I supply different…
insanely_sin
  • 986
  • 1
  • 14
  • 22
1
vote
0 answers

Enable ACL in Yarn application in Apache Twill v0.10.0

I'm running a yarn application using Apache Twill (0.10.0) in a HDP 2.5 cluster. I cannot see any container logs when I go to a container webpage: I suppose I need to set ACL from inside the application in the ContainerLaunchContext. Does Apache…
insanely_sin
  • 986
  • 1
  • 14
  • 22
0
votes
0 answers

Regenerating delegation tokens in long running yarn applications

I'm using Apache twill to deploy the yarn application. The app is a static hadoop application. As the title says, I couldn't figure out how to regenerate delegation tokens on expiry and ship it to all the containers/application master and make sure…
SphericalCow
  • 345
  • 1
  • 2
  • 13
0
votes
0 answers

Writing MapReduce and YARN application together

I want to run MapReduce application using Hadoop 2.6.5 (in my own native cluster) and I want to update some things in YARN thus I have seen that I can write my own YARN application…
Or Raz
  • 39
  • 2
  • 11
0
votes
1 answer

Does Apache Twill relaunch containers that are killed by Yarn?

Yarn kills containers when there is heavy load in the cluster. How does Apache Twill react when one of its runnablse running in the container gets killed? Does its run with reduced number of instances of the runnable or does it relaunch it?
insanely_sin
  • 986
  • 1
  • 14
  • 22
0
votes
1 answer

Apache Twill HelloWorld example fails

When I run Apache twill HelloWorld example as stated in http://twill.incubator.apache.org/GettingStarted.html I get this log: 10:44:47.888 [ STARTING] DEBUG o.a.twill.yarn.YarnTwillController - Yarn application status for HelloWorldRunnable …
Adam Horky
  • 13
  • 6
0
votes
1 answer

Unable to launch apache twill within hortonworks sandbox and eclipse

I can not get my development environment up and running a simple twill application within the hortonworks sandbox and eclipse. I can get everything to compile but I end up running into this error when I launch it within eclipse. 17:45:01.367 [IPC…
cesara
  • 842
  • 1
  • 7
  • 19
0
votes
1 answer

How to package and run twill sample application

I'm trying to use apache twill to build a YARN application. From the slides of twill presentation, they are talking about the use of maven-bundle-plugin to package the hello world sample. So to package the sample hello world, I first tried to…
bachr
  • 5,780
  • 12
  • 57
  • 92
0
votes
1 answer

Apache Twill HelloWorld application fails - jar not found

Has anyone run into problems running the HelloWorld Twill example? My Application gets accepted but then transitions to the "FAILED" state. Yarn application HelloWorldRunnable application_1406337868863_0013 completed with status FAILED The YARN…
Achaiah
  • 91
  • 1
  • 3
0
votes
1 answer

Using Janino in YARN with Apache Twill causes "Imported class x.y could not be loaded"

I'm porting an open source project, which uses Janino for dynamic compilation of classes, to YARN, using Apache Twill. This works great except one last error. When Janino is used with twill, I'm getting an exception that a class cannot be found,…
peter
  • 14,348
  • 9
  • 62
  • 96