1

I have a large graph (vertex# 9,634 and edges# 454,691). I am using shortest.paths() function to get the shortest path matrix.

But, unfortunately, I am getting an error, At vector.pmt:444 : cannot reserve space for vector, Out of memory.

I am using R version 3.6.3 (2020-02-29) Platform: x86_64-pc-linux-gnu (64-bit) Memory: 64 GB

Sample code is given below:

test <- read.csv("~/data.csv")
g1 <- test[1:5,]
graph1 <- graph_from_data_frame(g1, directed = FALSE)
E(graph1)$weight <- is.numeric(g1$weight)

mat_1 <- shortest.paths(graph1) # getting error in this line

I have gone through several posts and found one piece of information here but this code is not fulfilling my purpose. I need the whole matrix and then I have to do some filter in this matrix. For example, after getting the whole matrix, I will take only the cell values which have 2. Regarding filtering, I have another post in here.

Badly need your help and suggestion.

For your kind information: I am also trying to use 87GB in a cluster but getting the same error!

0Knowledge
  • 747
  • 3
  • 14
  • hi, interesting, not sure if this might be of interest https://stackoverflow.com/questions/66446633/apparently-not-justified-vector-pmt444-cannot-reserve-space-for-vector-out – IronMan Mar 09 '21 at 00:54
  • Hello, thanks. Yes, the same type of problems. No answer in this post! Moreover, the post you have comparing 2 different packages and but I am using only 1 package – 0Knowledge Mar 09 '21 at 00:57
  • I doubt that anyone can guess their way to a solution. Error conditions require better descriptions for good answers. That's why there's a specific close-vote justification that we can pick from a menu. – IRTFM Mar 09 '21 at 02:26
  • Just tried this with simulated data of the same size - it only used 2GB of ram and a few minutes of computing time. You'll need to provide a reproducible example to get any useful help I think. – pseudospin Mar 09 '21 at 02:28
  • @pseudospin could you tell me, how much RAM do you have? How you checked that it took only 2GB? – 0Knowledge Mar 09 '21 at 02:59
  • @IRTFM I am thinking about how can I give a good description. I know for getting an answer I have to describe clearly. But, the code is working fine for the small graph but showing the error for the large graph! – 0Knowledge Mar 09 '21 at 03:02
  • @Akib62 just check task manager or activity monitor. The rsession grabbed 2GB early on and it seemed to work within that. A 10000 by 10000 matrix uses about 800MB, and the shortest distance algorithm itself should use a negligible amount of memory on top of that. Try running with your data for increasing sizes of data till it breaks. Also, did you mean `as.numeric`? – pseudospin Mar 09 '21 at 09:07
  • I apologize, I have an error with my dataset. Actually, my dataset contains 4 columns instead of 3 columns and that's why I was getting errors. – 0Knowledge Mar 09 '21 at 17:25

0 Answers0