Use this tag for questions relevant to libhdf; a JNI based C api for Hadoop's DFS.
Questions tagged [libhdfs]
20 questions
0
votes
0 answers
Why some of HDFS reads are so slow, when all the data is cached in off-heap?
I'm doing performance test for HDFS. I'm using HDFS C++ API. I notice that most of the reads take up to 5 microseconds. But some of reads take 500-800 microseconds. What could be the reason, considering all the data is in OS cache? No disk access…
user1289
- 1,251
- 2
- 13
- 25
0
votes
2 answers
connecting pyarrow with libhdfs3
I'm trying to connect to a hadoop cluster via pyarrows' HdfsClient / hdfs.connect().
I noticed pyarrows' have_libhdfs3() function, which returns False.
How does one go about getting the required hdfs support for pyarrow? I understand there's a…
Jay
- 2,535
- 3
- 32
- 44
0
votes
1 answer
Hadoop Exception java.lang.NoSuchMethodError when calling hdfsOpenFile in hdfs C API
Firstly I had the hadoop-2.7.2 built using mvn and environment variables set correctly. Then I'm trying to run the simple example on Apache website: C API libhdfs example code
#include "hdfs.h"
int main(int argc, char **argv) {
hdfsFS fs =…
Bartholomew Kuma
- 1
- 2
0
votes
1 answer
libhdfs throws Illegal UTF8 string in constant pool error when running through callgrind
I am trying to run one my process through callgrind. One of the child process (that I need to trace) calls into libhdfs and when running through callgrind this throws an exception:
java.util.ServiceConfigurationError:…
Eqbal
- 4,722
- 12
- 38
- 47
-1
votes
1 answer
AppData\Local\Temp\cc59LXDc.o:test1.cpp: undefined reference to `constructNewObjectOfClass(JNIEnv_*, _jthrowable**, char const*, char const*, ...)'
Obijective : To access hadoop cluster file(HDFS read/write) through c++ code. Windows 7 operating system.
Below is my sample code
#include "hdfs.h"
#include "hdfs.cpp"
int main(int argc, char **argv) {
hdfsFS fs =hdfsConnect( "localhost" ,…
Anant Pukale
- 109
- 3
- 12