How to optimize this R script to use the minimum CPU and Memory possible

Question

I built this R script that generate a map and a background tiles, the problem is, I need to run it on PowerBI service, which has a very constrained resources (Ram and CPU), I attached a reproducible example

This example works fine in PowerBI service, but when I tried it with my real data only the raster or the map works, but when I do both, I get you exceed the resource available, and as it is not documented, I don't know if the issue is CPU or RAM.

what's the best way to profile this code and check which section to change

please notice the dataset is a raster saved as ASCII, using saveRDS, it is done outside PowerBI and loaded as a csv file, as PowerBI does not read binary data

# Input load. Please do not change, the dataset is generated by PowerBI, I change it only to have a reproducible example #
`dataset` = read.csv('https://raw.githubusercontent.com/djouallah/loadRobjectPBI/master/powerbidf.csv', check.names = FALSE, encoding = "UTF-8", blank.lines.skip = FALSE);
# Original Script. Please update your script content here and once completed copy below section back to the original editing window #
library(sf)
library(dplyr)
library(tmap)
library(tidyr)

tempdf <- dataset %>%
  filter(!is.na(Value))%>%
  dplyr::select(Index,Value)%>%
  arrange(Index)%>%
  mutate(Value = strsplit(as.character(Value), "@")) %>%
  unnest(Value)%>%
  dplyr::select(Value)


write.table(tempdf, file="test3.rds",row.names = FALSE,quote = FALSE,  col.names=FALSE)
rm(tempdf)
background <- readRDS('test3.rds', refhook = NULL)

dataset <- dataset[c("x","y","color","status","labels")]
dataset$color <- as.character(dataset$color)
dataset$labels <- as.character(dataset$labels)

map <- st_as_sf(dataset,coords = c("x", "y"), crs = 4326)

chartlegend <- dataset %>%
   dplyr::select(status,color)%>%
   distinct(status, color)%>%
   arrange(status)

rm(dataset)

tm_shape(background)+
           tm_rgb() +
           rm(background)+
           tm_shape(map) +
           tm_symbols(col = "color", size = 0.04,shape=19)+
           tm_shape(filter(map, !is.na(labels))) +
           tm_text(text="labels",col="white")+
           rm(map)+
           tm_add_legend(type='fill',labels=chartlegend$status, col=chartlegend$color)+
           tm_layout(frame = FALSE,bg.color = "transparent",legend.width=2)+
           tm_legend(position=c("left", "top"),text.size = 1.3)+
           rm(chartlegend)

If I understand this right, your script is fed (by PowerBI) an object like your `dataset` here, which is an encoded raster, which to get back to being a raster you write it out as a table using `write.table` which magically is the RDS representation of a raster object, such that `background` is now a raster? Because PowerBI has encoded the raster data in this format? — Spacedman, Dec 06 '19 at 15:30
actually I did, here is more context, https://datamonkeysite.com/2019/12/06/load-raster-tiles-to-powerbi-data-model/ — Mim, Dec 06 '19 at 15:33

score 1 · Answer 1 · answered Dec 07 '19 at 01:47

changing the code to use base R only did help a bit

# Input load. Please do not change #
`dataset` = read.csv('https://raw.githubusercontent.com/djouallah/loadRobjectPBI/master/powerbidf.csv', check.names = FALSE, encoding = "UTF-8", blank.lines.skip = FALSE);
# Original Script. Please update your script content here and once completed copy below section back to the original editing window #


library(sf)
library(tmap)

tempdf <-  dataset[dataset$Value!="",]
tempdf <- tempdf[c("Index","Value")]
tempdf <-  tempdf[order(tempdf$Index),]
tempdf <-  stack(setNames(strsplit(as.character(tempdf$Value),'@'), tempdf$Index))
tempdf <- tempdf["values"]
write.table(tempdf, file="test3.rds",row.names = FALSE,quote = FALSE,  col.names=FALSE)
rm(tempdf)
background <- readRDS('test3.rds', refhook = NULL)

dataset <- dataset[c("x","y","color","status","labels")]
dataset$color <- as.character(dataset$color)

map <- st_as_sf(dataset,coords = c("x", "y"), crs = 4326)


chartlegend <-     unique(dataset[c("status","color")])

rm(dataset)


tm_shape(background)+
tm_rgb() +
rm(background)+
tm_shape(map) +
tm_symbols(col = "color", size = 0.04,shape=19)+
tm_text(text="labels",col="white")+
rm(map)+
tm_add_legend(type='fill',labels=chartlegend$status, col=chartlegend$color)+
tm_layout(frame = FALSE,outer.margins = c(0.005, 0.6, 0.06, 0.005),bg.color = "transparent",legend.width=2)+
tm_legend(position=c("right", "top"),text.size = 1.3)+
rm(chartlegend)

How to optimize this R script to use the minimum CPU and Memory possible

1 Answers1