Stack Overflow
Stack Overflow

Questions tagged [spring-data-hadoop]

Spring for Apache Hadoop is an open-source project that provides unified configuration model and easy to use APIs for using HDFS, MapReduce, Pig, and Hive as well as developing and deploying YARN applications.

Spring for Apache Hadoop simplifies developing Apache Hadoop by providing a unified configuration model and easy to use APIs for using HDFS, MapReduce, Pig, and Hive. It also provides integration with other Spring ecosystem project such as Spring Integration and Spring Batch enabling you to develop solutions for big data ingest/export and Hadoop workflow orchestration.

Home page: http://projects.spring.io/spring-hadoop/

GitHub repo: https://github.com/spring-projects/spring-hadoop

47 questions
0
votes
1 answer

Spring boot AMQP and Spring Hadoop together ends up with missing EmbeddedServletContainerFactory bean

I have two small apps, one uses spring-boot-starter-amqp, other uses spring-data-hadoop-boot. I can run them separately without any problems. When I join them together, app start fails with exception:…
wst
  • 4,040
  • 4
  • 41
  • 59
0
votes
1 answer

Spring Data Hadoop and Hbase 1+

Since recently, hbase released the hbase 1.2, and there is a huge number of bugs fixed in hbase 1.0.0+, I am wondering if Spring is going to upgrade to use the new API.
Afshin Moazami
  • 2,092
  • 5
  • 33
  • 55
0
votes
3 answers

Customizing Yarn container

I'm testing spring-yarn integration API and I'm little confused about what is the best practice of Yarn container customization in terms of: 1) If I want to use spring-boot-yarn combo, what is the correct way of telling the spring boot to pick up my…
Tomas Kloucek
  • 251
  • 2
  • 12
0
votes
2 answers

How to filter records according to `timestamp` in Spring Data Hadoop?

I have a hbase table with a sample record as follows: 03af639717ae10eb743253433147e133 column=u:a, timestamp=1434300763147, value=apple 10f3d7f8fe8f25d5bdf52343a2601227 column=u:a, timestamp=1434300763148,…
Mubin Shrestha
  • 398
  • 3
  • 22
0
votes
1 answer

how can i use spring data for apache hadoop for cdh 5.4.x?

The official document note that "Spring for Apache Hadoop 2.2 GA" only support CDH 5.3.3 Is there a way to use "spring data hadoop" for CDH 5.4.X or i must wait for the next version ? Will the 2.3 version support CDH 5.4.x ? Edit: I've completed…
SieuCau
  • 195
  • 1
  • 2
  • 15
0
votes
1 answer

Spring Hadoop Samples Mapreduce throw auxService:mapreduce_shuffle exception

I am following spring hadoop samples here and after trying to run it under mapReduce project: $ mvn clean package $ sh ./target/appassembler/bin/wordcount I got this error: 17:42:55,067 INFO doop.yarn.client.api.impl.YarnClientImpl: 174 -…
user3006967
  • 3,291
  • 10
  • 47
  • 72
0
votes
1 answer

Spring data Hadoop, Hbase Rest API, HBase Java Client : which one would be the best to implement to handle communication between Android and HBase

Does anybody know what is the best method to communicate between HBase database and Android? Basically I want to do following from my Android app to HBase table: i. Insert data into it. ii. Query table and get data. iii. Update table. I had done…
Mubin Shrestha
  • 398
  • 3
  • 22
0
votes
0 answers

Can't run a HbaseTemplate demo

I am running HDP2.2 with Hbase 0.98.4.2.2.0.0 and HDFS 2.6.0.2.2.0.0, so I add the 2.1.2.RELEASE version of spring-data-hadoop to my maven pom file, but failed to complie because of "The type org.apache.hadoop.hbase.client.Scan cannot be resolved.…
Xiaobo Gu
  • 199
  • 1
  • 10
0
votes
1 answer

Errors while using both Spring Data Hadoop + Spring MVC

I just started with Spring and Hadoop. I had the following build.gradle which worked fine: dependencies { compile 'org.springframework:spring-webmvc:4.0.6.RELEASE' compile 'jstl:jstl:1.2' providedCompile…
celineu
  • 576
  • 1
  • 5
  • 18
0
votes
1 answer

Getting Spring-XD and the hdfs sink to work for maprfs

This is a question about spring-xd release 1.0.1 working together with maprfs, which is officially not yet supported. Still I would like to get it to work. So this is what we did: 1) adjusted the xd-shell and xd-worker and xd-singlenode shell…
user152468
  • 3,202
  • 6
  • 27
  • 57
0
votes
1 answer

Spring jdbchdfs job -

Currently the jdbchdfs job does not have partitionPath for output directory, like the hdfs sink. What is the recommended way for doing it? i don't also see any JIRA for this, any plan for such a thing be supported in future? I gave it a thought and…
Ali
  • 69
  • 8
0
votes
0 answers

why "spring-data-hadoop" need to make jar to avoid 'class not found exception'

I have read spring-data-hadoop reference docuements alot. However, I cannot understand this framework architecture(or usage, necessary) The most useful point I expected is programming map-reduce job on my local machine to run job on remote cluster.…
user3205568
  • 1
  • 1
0
votes
0 answers

Spring Boot & Hadoop: Protocol message tag had invalid wire type

I'm working on making a web server that will run MapReduce jobs, however I cannot seem to get the server to start. when I try to run the jar for the project, i get the following error java.io.IOException: Failed on local exception:…
god_of_the_raptors
  • 1
  • 2
0
votes
1 answer

Spring Yarn @OnContainerStart - how to invoke Mapper?

i'm using the Spring Yarn package with Spring Boot and i'm trying to figure out how i can start a Mapper from the @OnContainerStart event. how do i pass arguments to the mapper? how do i configure which mapper/reducer to use? i'm trying to follow…
incomplete-co.de
  • 2,137
  • 18
  • 23
0
votes
0 answers

How to configure MultiOutputFormat from HCatalog API in spring-hadoop project?

I was developing a hadoop application. And now I want to migrate it into a spring-hadoop project. The thing is in my mapreduce job. I was using MultiOutputFormat from the HCatalog API as I want to store information on multiple tables. And I can not…
AllenZheng
  • 26
  • 4
1 2
3
4