How can I reduce the RAM utilization in my program that use deeplearning4j (Memory-mapped files and WorkspaceConfiguration)?

Question

I'm using deeplearning4j but when i load pre-trained model for text-classification I don't have enough RAM on my pc.

I tried to change eclipse.ini file and add more memory changing Xms and Xmx. Unfortunately it doesn't work for me.

https://deeplearning4j.org/docs/latest/deeplearning4j-config-memory

In this link seems there is a possible solution to use less RAM even though it cost more time of corse, but I don't care now.

From that link:

Memory-mapped files ND4J supports the use of a memory-mapped file instead of RAM when using the nd4j-native backend. On one hand, it’s slower then RAM, but on other hand, it allows you to allocate memory chunks in a manner impossible otherwise.

Can I add this in a code like this (follow the link)?

https://github.com/deeplearning4j/dl4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/nlp/word2vec/Word2VecRawTextExample.java

Of cours if there is another way (or a better way) write it. I'll appreciate any advice.

Thanks in advance.

While I'm not familiar with this framework nor know whether `Word2Vec`, as used in the linked example, support memory-mapping, according to [some official documentation](https://deeplearning4j.org/docs/latest/deeplearning4j-config-memory) DL4J should be ready to deal with memory-mapped files and off-heap data. Best advice would be to just try it out and see if it works and probably ask the devs how you should approach what you are aiming for — Roman Vottner, Dec 07 '18 at 15:39

score 1 · Accepted Answer · answered Dec 08 '18 at 04:19

I'm from the deeplearning4j project. Memory mapped workspaces are made for embeddings yes and should be considered a separate concept from our off heap memory. The off heap memory is a conceptual rabbit hole I won't cover here (you have to have an understanding of the JVM and the topic isn't relevant here)

The way you would have to use memory mapped workspaces is by loading the word2vec inside a memory mapped scope. The first component is the configuration:

import org.nd4j.linalg.api.memory.MemoryWorkspace;
import org.nd4j.linalg.api.memory.conf.WorkspaceConfiguration;
import org.nd4j.linalg.api.memory.enums.LocationPolicy;
WorkspaceConfiguration mmap = WorkspaceConfiguration.builder()
            .initialSize(initialSize)
            .policyLocation(LocationPolicy.MMAP)
            .build();

try (MemoryWorkspace ws =   
           Nd4j.getWorkspaceManager().getAndActivateWorkspace(mmap)) {
 //load your word2vec here            

}

Of note with memory mapped workspaces is how it should be used. Mem map is intended only for accessing a large array and pulling subsets of it out from ram. You should only use it to pull out a subset of the word vectors out you need for doing training.

When using word2vec (or any other embedding technique), the typical pattern is to lookup only the word vectors you want and merge them together in to a mini batch. That minibatch (and the associated training) should happen in a separate workspace (or have it be unattached which is the default). The reason you can have it unattached is we already do workspaces and the other associated optimizations for you inside of ComputationGraph and MultiLayerNetwork. Just make sure to pass in whatever you need to fit.

From there, use the INDArray get(..) and put(..) methods to copy the rows you need in to another array that you should use for training. For more on that see: https://deeplearning4j.org/docs/latest/nd4j-overview

For more information look at leverage, leverageTo, detach,.. in the INDArray javadoc: https://deeplearning4j.org/api/latest/org/nd4j/linalg/api/ndarray/INDArray.html

How can I reduce the RAM utilization in my program that use deeplearning4j (Memory-mapped files and WorkspaceConfiguration)?

1 Answers1