3

I'm working on a C++ CMake project that uses Apache Arrow as a dependency. My goal is to be able to include and use arrow/api.h. However, I couldn't find any documentation or tutorial that explains what I can do to achieve that so my first thought was to add it as a third library using CMake's FetchContent, hence, I added the following code to my CMakeLists.txt:

include(FetchContent REQUIRED)

# INCLUDING APACHE ARROW ================================ #
message(STATUS "Searching for Apache Arrow")
FetchContent_Declare(
        arrow
        GIT_REPOSITORY https://github.com/apache/arrow.git
        GIT_TAG        apache-arrow-0.15.1
)
FetchContent_GetProperties(arrow)
if(NOT arrow_POPULATED)
    message(STATUS "Populating Arrow")
    FetchContent_Populate(arrow)
    add_subdirectory(${arrow_SOURCE_DIR}/cpp ${arrow_BINARY_DIR})
endif()

I can find the code inside my project now - still can't use include arrow/api.h though - but whenever I try to build it I receive errors from the CMakeLists.txt of Apache Arrow cpp src directory! Here's the CMake build log that contains the error:

-- Searching for Apache Arrow
-- Populating Arrow
-- Building using CMake version: 3.15.3
-- Arrow version: 0.15.1 (full: '0.15.1')
-- Arrow SO version: 15 (full: 15.1.0)
-- clang-tidy not found
-- clang-format not found
-- infer not found
-- Could NOT find PythonInterp (missing: PYTHON_EXECUTABLE) 
-- Found cpplint executable at CPPLINT_BIN-NOTFOUND
-- Compiler command: C:/MinGW/bin/g++.exe -v
-- Compiler version: Using built-in specs.
COLLECT_GCC=C:/MinGW/bin/g++.exe
COLLECT_LTO_WRAPPER=c:/mingw/bin/../libexec/gcc/mingw32/8.2.0/lto-wrapper.exe
Target: mingw32
Configured with: ../src/gcc-8.2.0/configure --build=x86_64-pc-linux-gnu --host=mingw32 --target=mingw32 --prefix=/mingw --disable-win32-registry --with-arch=i586 --with-tune=generic --enable-languages=c,c++,objc,obj-c++,fortran,ada --with-pkgversion='MinGW.org GCC-8.2.0-3' --with-gmp=/mingw --with-mpfr=/mingw --with-mpc=/mingw --enable-static --enable-shared --enable-threads --with-dwarf2 --disable-sjlj-exceptions --enable-version-specific-runtime-libs --with-libiconv-prefix=/mingw --with-libintl-prefix=/mingw --enable-libstdcxx-debug --with-isl=/mingw --enable-libgomp --disable-libvtv --enable-nls --disable-build-format-warnings
Thread model: win32
gcc version 8.2.0 (MinGW.org GCC-8.2.0-3) 

-- Compiler id: GNU
Selected compiler gcc 8.2.0
-- Arrow build warning level: CHECKIN
Configured for DEBUG build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})
-- Build Type: DEBUG
-- Using  approach to find dependencies
-- AWSSDK_VERSION: 1.7.160
-- BOOST_VERSION: 1.67.0
-- BROTLI_VERSION: v1.0.7
-- BZIP2_VERSION: 1.0.8
-- CARES_VERSION: 1.15.0
-- DOUBLE_CONVERSION_VERSION: v3.1.5
-- FLATBUFFERS_VERSION: v1.11.0
-- GBENCHMARK_VERSION: v1.5.0
-- GFLAGS_VERSION: v2.2.0
-- GLOG_VERSION: v0.3.5
-- GRPC_VERSION: v1.20.0
-- GTEST_VERSION: 1.8.1
-- JEMALLOC_VERSION: 5.2.1
-- LZ4_VERSION: v1.8.3
-- MIMALLOC_VERSION: 270e765454f98e8bab9d42609b153425f749fff6
-- ORC_VERSION: 1.5.5
-- PROTOBUF_VERSION: v3.7.1
-- RAPIDJSON_VERSION: 2bbd33b33217ff4a73434ebf10cdac41e2ef5e34
-- RE2_VERSION: 2019-08-01
-- SNAPPY_VERSION: 1.1.7
-- THRIFT_VERSION: 0.12.0
-- THRIFT_MD5_CHECKSUM: 3deebbb4d1ca77dd9c9e009a1ea02183
-- URIPARSER_VERSION: 0.9.3
-- ZLIB_VERSION: 1.2.11
-- ZSTD_VERSION: v1.4.3
-- Boost include dir: 
-- Boost libraries: 
-- Performing Test DOUBLE_CONVERSION_HAS_CASE_INSENSIBILITY
-- Performing Test DOUBLE_CONVERSION_HAS_CASE_INSENSIBILITY - Failed
-- Building without OpenSSL support. Minimum OpenSSL version 1.0.2 required.
-- Found hdfs.h at: D:/School/Research Project/TestApacheArrow/cmake-build-debug/_deps/arrow-src/cpp/thirdparty/hadoop/include/hdfs.h
-- CMAKE_C_FLAGS:  -g -O0   -Wall -Wno-conversion -Wno-sign-conversion -Wno-unused-variable -Werror
-- CMAKE_CXX_FLAGS:  -Wno-noexcept-type  -fdiagnostics-color=always -g -O0  -Wall -Wno-conversion -Wno-sign-conversion -Wno-unused-variable -Werror 
CMake Error at cmake-build-debug/_deps/arrow-src/cpp/src/arrow/CMakeLists.txt:362 (add_dependencies):
  add_dependencies called with incorrect number of arguments


CMake Error at cmake-build-debug/_deps/arrow-src/cpp/CMakeLists.txt:814 (export):
  export Export set "arrow-targets" not found.


-- ---------------------------------------------------------------------
-- Arrow version:                                 0.15.1
-- 
-- Build configuration summary:
--   Generator: MinGW Makefiles
--   Build type: DEBUG
--   Source directory: D:/School/Research Project/TestApacheArrow/cmake-build-debug/_deps/arrow-src/cpp
--   Install prefix: C:/Program Files (x86)/TestApacheArrow
-- Configuring incomplete, errors occurred!
See also "/cmake-build-debug/CMakeFiles/CMakeOutput.log".
See also "/cmake-build-debug/CMakeFiles/CMakeError.log".

Am I missing something here? is this the correct way to include Apache Arrow in an existing project? Any idea how can I fix this error?

I'm using Windows 10 but would finally want to run the project on Linux!

eyadMhanna
  • 2,412
  • 3
  • 31
  • 49
  • There appear to be several third-party dependencies that Apache Arrow requires. CMake complains that some of these have not been found (Python, infer, clang-tidy, etc.). Did you follow the **Developing for Windows** steps [here](https://github.com/apache/arrow/blob/master/docs/source/developers/cpp.rst#developing-on-windows)? Also, did you consider just installing the pre-compiled binaries as suggested [here](https://github.com/apache/arrow/tree/master/cpp#installation)? – Kevin Jan 15 '20 at 23:05

1 Answers1

2

We don't currently support including Apache Arrow in another CMake project using add_subdirectory. You can use ExternalProject_Add, though, and it should work. It should also be possible to use CMake's built-in find_package facility to find a pre-built package.

We would like to provide well-documented and straightforward solutions for C++ developers including Apache Arrow in their CMake projects. We'd be glad to discuss further on our developer mailing list dev@arrow.apache.org

Wes McKinney
  • 101,437
  • 32
  • 142
  • 108
  • Thank you, I added it using your advice, my CMakeLists file now looks like this include(ExternalProject) ExternalProject_Add(arrow URL "https://www.apache.org/dist/arrow/arrow-0.15.1/apache-arrow-0.15.1.tar.gz" SOURCE_SUBDIR cpp) However, I'm still not able to include the api.h as you did in your documentation C++ implementation example. It's saying arrow/api.h is not found. Any idea? – eyadMhanna Jan 30 '20 at 13:56