1

I've produced a Proof of concept to automate the export of data out of an external database and save it as a file to then import into ArangoDB using arangoimp.exe. While this method is certainly functional it is unfortunately not going to work due to my company not being able to utilize the file system in the external database or of the local file system that ArangoDB has access to.

The Question: Is there an alternative method or mechanism to import data from within ArangoDB?

Is there any inherent tool I can make use of such as:

I've read about both of these features in ArangoDB however I'm curious if either of these two features can do what I need.

I need to be able to automate from within ArangoDB a procedure/action that connects to an External Database and can then run SQL Queries OR run Stored Procedures in that external database and store that data directly into ArangoDB Collections.

Alternative: Do I need to code/develop my own program?

Many years ago I created a Win Forms app that can connect to several databases. Basically it was my first attempt at learning connection strings and sql injection. That project didn't end up going beyond just that but I've had thoughts in the back of my mind that are telling me I might have to develop an intermediary application to facilitate the data transfer I'm attempting to make happen.

My fear in the latter is that this just opens up a brand new project that needs to be maintained and developed internally which means resources will need to be devoted to it.

Cody Gray - on strike
  • 239,200
  • 50
  • 490
  • 574
Code Novice
  • 2,043
  • 1
  • 20
  • 44

1 Answers1

2

arangoimp has --server.endpoint parameter, which would allow one to import the data on a remote machine to an arangodb server. The two machines would just have to be on the same network.

Kaveh Vahedipour
  • 3,412
  • 1
  • 14
  • 22
  • I took a look at the --server.endpoint and it seems that parameter allows you to connect to an ArangoDB server other than the default 'localhost' 'tcp://127.0.0.1:8529' however arangoimp still requires a 'File'. Do you know if 'arangoimp' provides alternative methods of obtaining the data other than from a 'File'? Please let me know if I am missing something here. – Code Novice Mar 18 '18 at 21:21
  • Would your other db server have an export tool to an external machine? – Kaveh Vahedipour Mar 18 '18 at 21:22
  • The only method I've discovered is the ability to export data in the form of a file. I could open up the question further by asking: Are there any databases that have mechanisms built in that allow them to Access External Databases using a connection string or some other method to connect? Are there ETL tools available to do just this? A BI Connector for example does this for reporting but is there a tool that is designed to move and transform data without first saving it into a FILE on a file system? I would be surprised to find out that a tool like this does not yet exist. – Code Novice Mar 19 '18 at 16:43
  • You may or may not care however I'm hoping to pick your brain should you have further knowledge on this matter. I'm currently researching ETL tools. The one I've come across that appears promising is `Apache NiFi`. It has the ability to move data within a 'Process Flow' from Source to Destination. While it currently has 'Plug-ins' for MongoDB and MariaDB it has yet to implement ArangoDB. Another tool I want to check out is `Talend`. Currently I am unable to find nice tools like these that function with ArangoDB. Would have any ideas as to an ETL tool that works with ArangoDB? – Code Novice Mar 21 '18 at 17:47
  • I was able to get NiFi to get data into ArangoDB. and felt I'd share. I ended up answering one of my own questions and felt I'd add a link here: [ETL Tools for Arango](https://stackoverflow.com/questions/49436345/etl-tools-that-function-well-with-arangodb-what-are-they) – Code Novice Mar 23 '18 at 20:52
  • FWIW: `--file file name ("-" for STDIN)`, so if you specify the file name `"-"`, arangoimp will actually read from standard input instead of a file, which means you can pipe data to it. – CodeManX Apr 13 '18 at 13:04