0

Could you help to understand how can I load data from PostgreSQL into Deep Learning 4J, please? I'll appreciate if anyone have an example.

Thanks in advance

Celso Marques
  • 378
  • 1
  • 4
  • 15

2 Answers2

1

If you want to load your data from Postgres before vectorization, you can use the JdbcRecordReader. It is in the datavec-jdbc maven package.

For an example how to use it, check out the unit-test for it: https://github.com/eclipse/deeplearning4j/blob/master/datavec/datavec-jdbc/src/test/java/org/datavec/api/records/reader/impl/JDBCRecordReaderTest.java

This will give you access to your data in record form. If all your data is already numeric: great. If not, you will have to vectorize it. Explaining how to do that is probably too long for Stack Overflow. Take a look at https://www.dubs.tech/guides/quickstart-with-dl4j/ where I explain how to do it with records coming from a CSV file.

If you're confused beyond that, feel also free to ask questions on community.konduit.ai, there you can ask follow up questions better than on StackOverflow.

Paul Dubs
  • 798
  • 4
  • 8
0

dl4j, as any other framework works with tensors - INDArray in case of dl4j. So in order to "load data" you'll have to convert it to tensors actually. That applies to any data source - text, images, mp3s - everything is vectorized before sending into neural network.

raver119
  • 336
  • 1
  • 5
  • Ok, but is there any way to connect dl4j or nd4j directly to postgres? – Celso Marques Apr 01 '20 at 17:21
  • 1
    Sure no, there's literally no point in such a connection. That's what Data Engineers typically do - they convert raw data into something machine learning algorithm can understand. Plus they do data cleaning, pre/post processing, normalization etc etc etc – raver119 Apr 02 '20 at 10:01