0

I need help with an optimization and performance problem related to the calculation of the Weighted Geometric Mean of some data.

I introduce the problem with a little sample. I wrote the code to calculate the WGM for the simple example below.

% A matrix Example 3x3 matrix
% w column vector 3x1
% wgm row vector 1x3


A = rand(3);
w = [1,2,6]';
wgm = (prod(A.^w)).^(1/sum(w));

Now for the general problem:

Suppose I have a new A matrix sized nxm and a W matrix composed of weight columns where weights values can go from 0 to k and I need all the columns permutations.

That is the W matrix is sized as n x k^n, since the nature of the weights and the weighted geometric computation this final matrix should be reduced excluding columns that represent a multiplication by a scalar value going from 0 to k of a permutation.

So if I have a column like [1, 1, 0] already that should exclude all t*[1,1,0] with t going from 0 to k. Another example: [1 2 3] should exclude [2 4 6] or [3 6 9] and so on.

Basic idea: each generated column for the W matrix could be normalized dividing each weight by k, so if the new normalized column is redundant should not be added then converted back to an uint8 column to reduce memory consumption to 12.5%.

So considering a real data example suppose:

  1. I have a static A matrix 32x30.
  2. Weights values that go from 0 to 99.
  3. I need a way to create the W Matrix sized 32x100^32 and to optimize it.
  4. To calculate an optimized originally WGM 100^32x30 matrix where each row is the computation result from A matrix and corresponding W column.

So the problems to solve are:

  • Creation of the optimized matrix of Weights both in size and performance.
  • Calculation of the WGM Matrix.
  • A way to allocate and partition those matrixes to avoid memory problems.
  • Converting the Matlab code to GPU code for computation on a Cuda Device (1080 GTX with 8 GB video memory).
  • Storing the final matrices in an efficient way.

Added information:

The Weighted Geometric Means matrix data will be validated through a set of stricter rules, and so not compliant rows will be discarded, same for the final W matrix where the elimination will occur for corresponding columns.

This could be evaluated earlier while creating the 2 matrices to find a solution that optimizes memory consumption while being maybe less efficient performance wise.

Yoni
  • 1,346
  • 3
  • 16
  • 38
Relok
  • 301
  • 4
  • 14
  • What is the question? – Ander Biguri Apr 24 '18 at 19:04
  • Question is how to write an optimized solution that takes into account the problems listed in the end with the first algoritm as a codebase with modifications needed to deal expecially with the memory, I've no Idea on how to start optimizing this apart what i listed, huge memory seems to be required to handle it. Also converting to code for parallel computing is a problem too (since I never did anything like this) and I think the data must be prepared to work correctly for using it on the GPU. – Relok Apr 24 '18 at 19:44
  • You are just asking how to do everything, or for a tutorial, none of them an appropiate question in stackoverflow – Ander Biguri Apr 24 '18 at 20:07
  • I don't need the full solution to the problem, those are the problems I need to solve, If I get some help I can move forward, and so I'll eventually figure out other steps while I dig the problems further. I detailed all the problems so I can get more precise replies avoiding people not to fully understand the problems I'm facing or having answers that will be good for a problem and at the same time bad for another. – Relok Apr 24 '18 at 21:39

0 Answers0