-1

Obijective : To access hadoop cluster file(HDFS read/write) through c++ code. Windows 7 operating system. Below is my sample code 

#include "hdfs.h"
#include "hdfs.cpp"
int main(int argc, char **argv) {
hdfsFS fs =hdfsConnect( "localhost" , 9000);
return 0
}

NOTE: I have single node hadoop cluster up and running.

g++ test1.cpp

Getting below error stack trace for above command:

C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x60): undefined r
eference to `constructNewObjectOfClass(JNIEnv_*, _jthrowable**, char const*, cha
r const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0xe1): undefined r
eference to `classNameOfObject(_jobject*, JNIEnv_*)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x242): undefined
reference to `hdfsConnectAsUser'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x296): undefined
reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x2d7): undefined
reference to `constructNewObjectOfClass(JNIEnv_*, _jthrowable**, char const*, ch
ar const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x38b): undefined
reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject*
, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x437): undefined
reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject*
, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x4cc): undefined
reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject*
, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x5e4): undefined
reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject*
, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x671): undefined
reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject*
, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x6f8): undefined
reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x77f): undefined
reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject*
, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x7e5): undefined
reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x95e): undefined
reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject*
, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0xa62): undefined
reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject*
, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0xb0f): undefined
reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject*
, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0xba0): undefined
reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject*
, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0xc45): undefined
reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject*
, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0xce9): more undef
ined references to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jo
bject*, char const*, char const*, char const*, ...)' follow
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0xe53): undefined
reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0xedd): undefined
reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0xf74): undefined
reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject*
, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0xff2): undefined
reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x10fd): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x11d8): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x1300): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x13cf): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x1522): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x1587): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x1636): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x167f): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x1710): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x17ae): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x17f8): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x189f): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x18ed): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x19ab): undefined
 reference to `constructNewObjectOfClass(JNIEnv_*, _jthrowable**, char const*, c
har const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x1a8c): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x1ad5): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x1b93): undefined
 reference to `constructNewObjectOfClass(JNIEnv_*, _jthrowable**, char const*, c
har const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x1c74): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x1cbd): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x1d54): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x1dd2): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x1eb5): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x1f57): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x1fc9): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x2057): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x212b): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x21ca): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x2224): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x22bf): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x231a): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x23b9): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x240d): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x252f): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x257e): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x25d5): undefined
 reference to `constructNewObjectOfClass(JNIEnv_*, _jthrowable**, char const*, c
har const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x267c): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x26e9): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x2818): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x28aa): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x2948): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x2a0d): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x2b48): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x2e40): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x2ebe): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x2f1a): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x2f56): undefined
 reference to `globalClassReference(char const*, JNIEnv_*)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x2ff6): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x3045): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x3081): undefined
 reference to `globalClassReference(char const*, JNIEnv_*)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x3121): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x31b2): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x3241): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x32c2): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x3347): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x33e7): more unde
fined references to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _j
object*, char const*, char const*, char const*, ...)' follow
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x3ade): undefined
 reference to `getJNIEnv()'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x3b83): undefined
 reference to `invokeMethod(JNIEnv_*, jvalue*, _jthrowable**, MethType, _jobject
*, char const*, char const*, char const*, ...)'
C:\Users\anant\AppData\Local\Temp\cc59LXDc.o:test1.cpp:(.text+0x3ce6): undefined
 reference to `getJNIEnv()'
collect2.exe: error: ld returned 1 exit status
Jens Gustedt
  • 76,821
  • 6
  • 102
  • 177
Anant Pukale
  • 109
  • 3
  • 12

1 Answers1

0

Did you install the library ?

Add linker settings and add linker search directories.

Rafael
  • 49
  • 1
  • 8
  • I am doing C++ code first time. i have g++ installed on windows 7 64 but machine. All i did is i have single node hadoop up and running and tried to execute g++ test1.cpp. kindly help with steps stiil required to be taken. – Anant Pukale Jun 06 '18 at 04:46
  • This is for an older version but the implementation technique should not change. http://www.science.smith.edu/dftwiki/index.php/Hadoop_Tutorial_2.2_--_Running_C%2B%2B_Programs_on_Hadoop#Compiling_and_Running – Rafael Jun 06 '18 at 06:30
  • ,I tried to follow ur link . Getting linking error : /almdev/hadoop/wordcount.cpp:57: undefined reference to `HadoopPipes::runTask(HadoopPipes::Factory const&)' and my Makefile looks like this : CC = g++ HADOOP_INSTALL = /almdev/hadoop/hadoop PLATFORM = Linux-i386-32 CPPFLAGS = -m32 -I$(HADOOP_INSTALL)/c++/$(PLATFORM)/include wordcount: wordcount.cpp $(CC) $(CPPFLAGS) $< -Wall -L$(HADOOP_INSTALL)/c++/$(PLATFORM)/lib \ -L$lib/native/hadooppipes.a -L$lib/native/hadooputils.a -lpthread -g -O2 -o $@ ...@Rafael – Anant Pukale Jun 07 '18 at 04:58
  • appending -lcrypto resolved above issue to create binary executable.@Rafael ... according your above link the execution command with hadoop pipes is no longer supported. suggested new command with mapred is also no longer in use. any other alternate solution ? @Rafael – Anant Pukale Jun 07 '18 at 07:53
  • In local mode is not supported, since Hadoop pipes can run only in distributed or pseudo distributed mode. – Rafael Jun 08 '18 at 06:22